1. Predictive Analytics
1.1. ...we approach the challenge of predicting future values in a number of ways...
1.1.1. Time Series Forecasting
1.1.1.1. ...basic forecasts can be achieved in Power BI using built-in features...
1.1.1.1.1. Microsoft: Forecasting Tool
1.1.1.2. ...more...
1.1.1.2.1. ESRI: Time Series Forecasting 101
1.1.1.2.2. Towards Data Science: The Complete Guide to Time Series Analysis and Forecasting
1.1.1.2.3. Machine Learning Mastery: What Is Time Series Forecasting?
1.1.2. Supervised Machine Learning
1.1.2.1. ...regression is typically the first port of call in the majority of supervised machine learning projects...
1.1.2.1.1. Types of Machine Learning
1.1.2.1.2. Algorithms of Machine Learning
1.1.2.2. ...this is because it offers an open and intuitive equation for development and use...
1.1.2.2.1. Regression Equation
1.1.2.3. ...we can easily identify regression-based projects by the conceptualisation of a system as having "dependent" outputs being driven by "independent" inputs...
1.1.2.3.1. SMU Default Prediction
1.1.2.3.2. Deloitte Predictive Maintenance
1.1.2.3.3. Input-Output Prediction
1.1.2.4. ...we can craft basic regression models in DAX...
1.1.2.4.1. Regression Concept in DAX
1.1.2.5. ...however, we may prefer to adopt alternative tools to gain better control over the "process" of machine learning...
1.1.2.5.1. Machine Learning Process
1.1.2.5.2. ML in KNIME and Power BI
2. Advanced Data Visualisation
2.1. ...data visualisation is a core component of data science, it uses four principal insights to tackle four principal challenges...
2.1.1. Four Principal Insights
2.1.1.1. ...more...
2.1.1.1.1. 10 rules for better dashboard design
2.1.2. Four Principal Challenges
2.2. ...data visualisation is a deep science in its own right, with a wide range of specialist theorists and commentators...
2.2.1. Stephen Few: show insight, simplify, clarify, and generate density
2.2.2. Alberto Cairo: tell stories, enable the user to take their own path
2.2.3. Edward Tufte: show the data, not the ducks, do not mislead...
2.2.3.1. ...charts can be sources of truth, lies, and misunderstandings...
2.2.3.1.1. How to Lie with Statistics: Stand Your Ground and Gun Deaths
2.2.3.1.2. John Snow and Cholera
2.3. ...in Power BI we can use the core "charting" tools in combination with DAX to address the classic challenges of data science...
2.3.1. DAX Fundamentals
2.3.1.1. ...we can start by familiarising ourselves with simple fragments of DAX...
2.3.1.1.1. DAX Practice with the Adventure Works Report
2.3.1.2. ...we can extend this understanding of DAX to cover a set of common challenges...
2.3.1.2.1. Illustrative DAX Scenarios
2.3.1.3. ...in reality, DAX solutions typically come from pattern books or forum contributions...
2.3.1.3.1. DAX Patterns
2.3.2. DAX and Visualisation
2.3.2.1. ...we can use DAX to support our visual data science work, focussing on the four principal visual data science insights...
2.3.2.1.1. Relationship
2.3.2.1.2. Composition
2.3.2.1.3. Distribution
2.3.2.1.4. Comparison
2.4. ...when we begin to reach our limit with DAX, we might find a solution to our challenge in Python code...
2.4.1. Leveraging Python
2.4.1.1. ...in some cases we can use Python visualisation libraries to deepen and extend our insight...
2.4.1.1.1. Preparation
2.4.1.1.2. Matplotlib
2.4.1.1.3. Seaborn
2.4.1.2. ...when using Python in Power BI, we should bear in mind the "pro-con"...
2.4.1.2.1. Pro: breadth of sophisticated visuals from the data science domain, enabling me to find and understand hidden features in datasets
2.4.1.2.2. Con: not all Python libraries are present in Power BI Online, datasets must be smaller than 150k rows, and visuals are not interactive
3. Text Analytics
3.1. ...pulling quantitative insight from text and language is a key thrust of modern analytics work...
3.1.1. Text Analytics in Algorithmic Trading
3.1.1.1. ...more...
3.1.1.1.1. VADER Sentiment Analysis in Algorithmic Trading
3.2. ...a wide range of use cases can be imagined...
3.2.1. Zencos Use Case Overview
3.2.1.1. ...more...
3.2.1.1.1. Text Analytics: 5 Examples To Open Your Eyes To Your Own Opportunities
3.3. ...the types of insight we might pull from text can be quite varied, we will focus on two techniques...
3.3.1. Sentiment Analysis: the process of systematically identifying, extracting, and quantifying affective states and subjective information in text or language
3.3.1.1. ...sentiment analysis assesses the content of identifiers in text and determines an overall "polarity" score for that text or its components...
3.3.1.1.1. Identifiers and Polarity
3.3.1.2. ...sentiment analyses feature prominently in tools tailored to understand complex human scenarios...
3.3.1.2.1. MonkeyLearn - Chewy Trustpilot Reviews
3.3.1.3. ...we can use insight into sentiment for a range of purposes...
3.3.1.3.1. Trading Signals
3.3.1.3.2. Customer Care
3.3.2. Topic Modelling: a type of statistical modeling used to discover the abstract “topics” that occur in a collection of documents
3.3.2.1. ...topic models group texts which share content or identifies words and phrases that occur together frequently...
3.3.2.1.1. Topic Discovery and Clustering
3.3.2.2. ...we can use topic models to generate summary insights into large volumes of material...
3.3.2.2.1. Document and Contract Review
3.3.2.2.2. Social Listening
3.3.2.3. ...we can blend sentiment analysis and topic modelling to automate or amplify insight and discovery initiatives...
3.3.2.3.1. Opinion Mining
3.3.2.4. ...we can get a sense of what a topic model is doing by looking at the OpenFraming tool...
3.3.2.4.1. Use OpenFraming.ai on the Trump Tweets dataset...
3.4. ...these techniques require the text to be pre-processed before the analytics can be performed, this is the route of the "Natural Language Processing", or "NLP", term...
3.4.1. NLP Overview by Lexalytics
3.4.1.1. ...a common set of preparatory tasks can be seen across recommended NLP workflows and frameworks...
3.4.1.1.1. Tokenization: breaking a string of text data into smaller "token" units that make up words or phrases
3.4.1.1.2. Part-of-Speech-Tagging: assigning each token in the data a grammatical label (i.e. noun, verb, adjective, or adverb)
3.4.1.1.3. Chunking and Parsing: deriving the sentence architecture and syntactical structure of the text data
3.4.1.1.4. Lemmatisation and Stemming: removing the suffixes, prefixes, and affixes of associated tokens to return their core dictionary form
3.4.1.1.5. Stopword Removal: removing tokens that have frequent occurrence but carry no meaning or value in the text (e.g. "and", "the", and "a")
3.4.1.2. ...once complete, the workflow can progress to the analytics stage...
3.4.1.2.1. Data Science Central Workflow
3.4.1.2.2. Tibco Text Analytics Workflow
3.5. ...we can use Python and KNIME to bring this capability into Power BI...
3.5.1. Power BI and VADER
3.5.1.1. ...a simple sentiment analysis...
3.5.1.1.1. Connect Power BI and Python
3.5.1.1.2. Prepare Python for Power BI
3.5.1.1.3. Connect to "tweets.csv" from Power BI
3.5.1.1.4. Implement a VADER Script
3.5.1.1.5. Interpret the VADER Results
3.5.1.1.6. Modify the VADER Script
3.5.1.1.7. Visualise the VADER Results
3.5.1.2. ...an applied scenario...
3.5.1.2.1. Craft a News API "Get"
3.5.1.2.2. Connect to the News API from Power BI
3.5.1.2.3. Repeat the Sentiment Analysis Steps
3.5.1.2.4. e856d1b8f83d4586ba558335a9ab9b7a
3.5.2. Power BI and PyCaret
3.5.2.1. ...advanced topic modelling is feasible in Power BI using Python...
3.5.2.1.1. Topic Modeling in Power BI using PyCaret
3.5.2.2. ...but there are limitations to Python's capabilities in Power BI...
3.5.2.2.1. Requirements and Limitations of Python Packages
3.5.2.3. ...and the complexities of advanced topic modelling tend to result in the analytics being pushed upstream in the pipeline...
3.5.2.3.1. Choosing a natural language processing technology in Azure
3.5.3. Power BI and KNIME
3.5.3.1. ...KNIME is a data science platform that allows us to design, build, and manage complex analytics pipelines upstream of Power BI...
3.5.3.1.1. Using the new KNIME Deep Learning - Keras Integration to Predict Cancer Type from Histopathology Slide Images
3.5.3.2. ...KNIME provides low code solutions to complex data science challenges and technology hurdles...
3.5.3.2.1. Data Science Presents Tech Hurdles
3.5.3.3. ...it is representative of the data science and machine learning platforms that are becoming common in advanced analytics solutions...
3.5.3.3.1. Components of a Big Data Architecture
3.5.3.3.2. Modern Analytics Architecture
3.5.3.3.3. Big Data Architectural Pattern
3.5.3.4. ...KNIME is a good option because it is an industry leader and accessible via open-source licensing...
3.5.3.4.1. KNIME in the Market
3.5.3.5. ...KNIME is straight-forward to use...
3.5.3.5.1. Driving Analytics with KNIME
4. Dashboard Development
4.1. ...Power BI is first and foremost a dashboard building solution, we can take an advanced dashboard scenario to recap and develop our understanding of key concepts...
4.1.1. Advanced Dashboard Scenario
4.1.1.1. ...the Power BI workflow consists of seven steps...
4.1.1.1.1. Step #0: Design
4.1.1.1.2. Step #1: Query
4.1.1.1.3. Step #2: Wrangle
4.1.1.1.4. Step #3: Model
4.1.1.1.5. Step #4: Metrics
4.1.1.1.6. Step #5: Visualise
4.1.1.1.7. Step #6: Publish
4.1.1.1.8. Step #7: Manage
5. Course Notes
5.1. ...course...
5.1.1. Data Analytics Using Advanced Power BI
5.2. ...materials...
5.2.1. http://bit.ly/SMU_ABI
5.2.1.1. ...more...
5.2.1.1.1. Copy Public Map
5.3. ...tools...
5.3.1. Power BI Desktop
5.3.2. Power BI Online
5.3.3. WinPython
5.3.4. KNIME
6. Framing
6.1. ...visualisation is a translational skill built on a foundation of solid engineering...
6.1.1. McKinsey: Analytics Translators
6.1.1.1. ...more...
6.1.1.1.1. McKinsey: Ten Red Flags Signaling Your Analytics Program Will Fail
6.1.1.1.2. Uber: Scaling Machine Learning at Uber with Michelangelo
6.2. ...we need to maintain a strong data science discipline to craft meaningful answers to important questions...
6.2.1. Tibco: Data Science