In this episode of TFiR Let’s Talk, Swapnil Bhartiya sits down with Gaetan Castelein, VP of Marketing at Tecton, to discuss the trend towards real-time machine learning and how Tecton is at the forefront of helping organizations leverage this for their services and applications.
Key highlights of this video interview:
- Majority of machine learning is still running in batch and is mainly used to complement existing analytics processes that populate dashboards for decision-makers to review. Meaning, the predictions and decisions are done at human speed and scale.
- Castelein is seeing a rapid increase in the number of enterprises embarking on this journey of using real-time ML. He believes this is due to better tools, like Tecton, and data scientists complemented ML engineers who are getting better at these processes.
- Use cases well-suited to this type of ML include those that need real-time decisions (e.g., loan underwriting, insurance policy underwriting) and where preferences can be curated into recommendations or generating ETAs (e.g., food delivery).
- Recommender systems are a subset of machine learning use cases, of which Spotify, Netflix, and Amazon are examples. Castelein explains why these systems are inherently more complicated and the data aspects can be particularly challenging.
- A recommender system requires the integration of lots of different data sources in order to come up with a prediction. It is more effective if it takes into account both real-time and historical data.
- Tecton hosts this year’s apply(recsys), a virtual conference on data engineering for machine learning recommender systems. Featured speakers have operationalized real-time machine learning and will share their experiences, best practices, tools of choice, architectures that work best, how to organize a team, etc.
The summary of the show is written by Emily Nicholls.