The Way This Work Unfolds

The way this work unfolds is often just through sheer aggregation. As you collect more and more examples, you begin to build patterns in your mind that emerge as intuition. Much of what people do in statistical learning is to recreate algorithmically what the brain does on its own. It often requires much more data, but that also means that claims made about the patterns are substantiated by the scale of the exercise. Nonetheless, you can make perfectly good claims based on data at a smaller scale. Do not be afraid to lean into your intuitions. Also, do not be disappointed if your first intuitions do not pan out. Inevitably, if you press on with data collection, more patterns emerge. Often those patterns and insights are the better ones because they did not come at first, when you are more likely to see what you want to see. Rather, coming later, they came in spite of you not seeing them. This is their strength.

Leave a Reply