Major Canadian City brings its data vision to life with a cloud-based Microsoft Azure solution

The challenge

Motivated by the desire to maintain and improve its global rating, a major Canadian City had invested in technology and innovation to support its growth while continuing to provide high levels of service. It had an ambitious vision to advance its current technology capability and enable the organization through strong data analytics with a platform that could scale with growth to meet its current and future needs. The goal was to simplify delivery, increase quality, and reduce the time to value for data reporting and analytics to provide a higher level of service to the City’s citizens and visitors. 

Additionally, the City was currently storing its data on-premises with limited architecture infrastructure. Its data team performed an assessment of the tools available on the market and entered discussions with Microsoft to begin the process of identifying the ideal data platform to fit the City’s current and future needs.

However, the City did not have the in-house capacity and skills it needed to rapidly build the new data infrastructure — and after an in-depth procurement process, they selected MNP Digital to support the process.

The approach

MNP Digital kickstarted the process by working with the City’s data team and software vendors to establish a proof of concept (PoC) for the new data platform. The purpose of the PoC was to demonstrate how a modern data and analytics platform could support the City’s goals and bring greater business value to key stakeholders within the organization.

MNP Digital built an Azure cloud-based solution, which leveraged Databricks Unity Catalog and a metadata-driven approach. The PoC used the City’s existing Claims-related data to demonstrate an improved process for data discovery and management that could be applied to all areas of the organization.

To support the management of the data and information within the platform, the Informatica platform was also implemented to give the City greater governance over its data. Informatica’s Data Profiling can be used to identify the common dimensions of a quality such as completeness by executing a profile run on a dataset. Data profiles also display unique values, patterns, and frequencies of values to support the data steward in analyzing their data.

Informatica’s Data Governance and Catalog platform also provides the ability to create business terms and policies in addition to the technical metadata, which can be attached or associated to technical assets. This helps business users navigate data in a language that is familiar to them and supports greater governance over the City’s data.

The benefits of the solution include:

• Enabling the City’s data team to apply a dynamic blueprint which leverages control tables. It accomplished this by establishing a platform based on a medallion architecture with a metadata pipeline for populating the data. This helped streamline data processing, data quality, data product creation, and analytic workflows in a scalable manner.
• Integration with Azure Log Analytics to help the City’s data team effectively monitor and analyze data logs, providing a comprehensive and granular view of all activities.
• Integration with Microsoft Power BI to create reports from the different data catalog categories stored in Databricks using dataflows and shared datasets.

In collaboration with the City, MNP Digital provided thought leadership and technical guidance on establishing governance councils and committees. This is required to ensure the long-term health and longevity of both the platform and processes within the City.

The result

MNP Digital worked with the City’s data team to develop a data platform that enables the organization to collect, process, and manage data effectively. The PoC also provided strong governance tools to identify data quality, data classification, and data access workflows. The PoC has now been deployed to production and additional organizations within the City have been onboarded onto the platform — with more planned throughout the year.

The new data platform also respects the City’s existing security measures, processes data entirely within the City’s subscription, and reduces vendor lock-in. Additionally, leveraging the cloud enables the data platform to scale with demand to allow for continued growth and expansion.

Working together with MNP Digital to design new data architecture has provided a solid foundation that will enable the City to leverage data into more impactful initiatives. This includes optimizing service delivery and enhancing efficiency to keep up with the demands of citizens.

Additionally, the City’s data team shifted its mindset during this engagement to become more outcome-focused. This includes:

• Developing an understanding of what problem the data needs to solve
• Understanding outcomes of implementing a dashboard for different business units within the municipality

This new mindset helps ensure data will be used to support the needs of the business as well as the needs of product development and delivery.

The City’s data team says that working with MNP Digital has given them the right capabilities and conditions for success in the future. The data team is now working develop new products based on the infrastructure they put into place with MNP Digital to help business units meet their needs and objectives. This infrastructure also enabled the City’s data team to make informed decisions from centralized data and accelerate the creation of products or business operations — setting them up for success today and tomorrow.