The ML Mindset
It's about Experimental Design
In traditional software engineering, you can reason from requirements to a workable design, but with machine learning, it will be necessary to experiment to find a workable model.
Many machine learning systems produce models that encode knowledge and intelligence by interpreting signals differently than humans do. A neural network might interpret a word via an embedding, so "tree" is understood as something like, [0.37, 0.24, 0.2] and "car" as [0.1, 0.78, 0.9]. The neural network might use these representations to do accurate translations or sentiment analysis, but a human looking at the embeddings would find them very hard to understand. This can make machine intelligence difficult, but not impossible, for humans to understand and evaluate.
Models will make strange mistakes that are difficult to debug, due to anything from skewed training data to unexpected interpretations of data during training. Furthermore, when machine-learned models are incorporated into products, the interactions can be complicated, making it difficult to predict and test all possible situations. These challenges require product teams to spend a lot of time figuring out what their machine learning systems are doing and how to improve them.
Get Comfortable with Some Uncertainty
Beyond simply thinking about problems differently, implementing ML is different than traditional programming. In traditional programming, you have set parameters and you understand how everything should behave. With ML, the non-coding work can be very complicated, but you'll usually write far less code.
Will you end-up with a usable model? You don't really know at the start.
Scientific Method
To address the challenges of transitioning to ML, it is helpful to think of the ML process as an experiment where we run test after test after test to converge on a workable model. Like an experiment,the process can be exciting, challenging, and ultimately worthwhile.
Step | Example |
|---|---|
1. Set the research goal. | I want to predict how heavy traffic will be on a given day. |
2. Make a hypothesis. | I think the weather forecast is an informative signal. |
3. Collect the data. | Collect historical traffic data and weather on each day. |
4. Test your hypothesis. | Train a model using this data. |
5. Analyze your results. | Is this model better than existing systems? |
6. Reach a conclusion. | I should (not) use this model to make predictions, because of X, Y, and Z. |
7. Refine hypothesis and repeat. | Time of year could be a helpful signal. |
Portions of this page are modifications based on work created and shared by Google and used according to terms described in the Creative Commons Attribution 4.0 License.