Hi @Sujitha
Just to follow up on your suggestion to share my takeaways from Jonathan Frankel's talk at Sigma in NYC. The key ideas I came away with is:
- Building in-house custom models is more than just possible, there's advantages to it
- There's danger in many ways due to the possibility of increasing in building and training, without ROI
- The prediction that ML models will start to diversify and Industry Domain specific models will start emerging with companies specializing in specific areas (e.g. Auto insurance claims processing models, CPG predictive inventory analysis and ordering models in specific market verticals)
And, I think the key point stressed over and over, is that Jonathan and Databricks will help companies and organizations to build these models in house, using their data and advise them on how to not over-spend and to overcome certain pitfalls (i.e. over-training, throwing good dollars after bad when a model goes off the rails).
Not sure if this next talk will be as specific, but I think there's key insights to what's going on at the forefront of the industry. Usually, what we hear and read about is only the exhaust trails of what is going on in the front of the pack. I think hearing from people like Jonathan, or other orgs/ppl like Srinivas from Perplexity is a great way to see what possibilities are coming down the pipeline. Not, that they will all realize fruition! 😊
#dbfs
