Paul Krill
Editor at Large

Google Prediction Framework addresses data pipeline drudgery

news
Jan 5, 20222 mins
AnalyticsCloud ComputingGoogle Cloud Platform

Prediction Framework weaves together Google Cloud Platform services to simplify the implementation of prediction projects.

executive pushing boulder uphill deadline stress struggle
Credit: Thinkstock

Googleโ€™s Prediction Framework stitches together Google Cloud Platform services, from Cloud Functions to Pub/Sub to Vertex AutoML to BigQuery, to help users implement data science prediction projects and save time doing so.

Detailed in a December 29 blog post, Prediction Framework was designed to provide the basic scaffolding for prediction solutions and allow for customization. Built for hosting on the Google Cloud Platform, the framework is an attempt to generalize all steps involved in a prediction project, including data extraction, data preparation, filtering, prediction, and post-processing. The idea behind the framework is that with just a few particularizations/modifications, the framework would fit any similar use case, with a high level of reliability.

Code for the framework can be found on GitHub. Prediction Framework uses Google Cloud Functions for data processing, Vertex AutoML for hosting the model, and BigQuery for the final storage of predictions. Google Cloud Firestore, Pub/Sub, and Schedulers are also used in the pipeline. Users must provide a configuration file with environment variables about the cloud project, data sources, the ML model, and the scheduler for the throttling system.

In explaining the frameworkโ€™s usefulness, Google noted that many marketing scenarios require analysis of first-party data, performing predictions on data, and leveraging results in marketing platforms such as Google Ads. Feeding these platforms regularly requires a report-oriented and cost-reduced ETL and prediction pipeline. Prediction Framework helps with implementing data prediction projects by providing the backbone elements of the predictive process.

Paul Krill

Paul Krill is editor at large at InfoWorld. Paul has been covering computer technology as a news and feature reporter for more than 35 years, including 30 years at InfoWorld. He has specialized in coverage of software development tools and technologies since the 1990s, and he continues to lead InfoWorldโ€™s news coverage of software development platforms including Java and .NET and programming languages including JavaScript, TypeScript, PHP, Python, Ruby, Rust, and Go. Long trusted as a reporter who prioritizes accuracy, integrity, and the best interests of readers, Paul is sought out by technology companies and industry organizations who want to reach InfoWorldโ€™s audience of software developers and other information technology professionals. Paul has won a โ€œBest Technology News Coverageโ€ award from IDG.

More from this author