Skip to content
  • About Us
  • Our Services
  • Case Studies
  • Content Hub
  • Blog
  • Join Us
  • Contact Us
Payments Industry
Daniel Hurst

The Future of Payments: Live-Streaming Mainframe Data to the Cloud

From the 90s, the payments industry made billions ($$$!) off the back of first-class reliability and resilience.

And they bet all of their chips on making transactions safe and secure.

But modern tech is pushing the source of value beyond just reliability and resilience.

And these payment providers haven’t always been investing elsewhere! Meanwhile, some serious competition (from innovators and Big Tech) is progressing at breakneck speed, creating a fluid transaction ecosystem and churning out ever-evolving sets of products and tech talent…

But there is a solution! Payment providers are sitting on a DATA GOLD MINE in their mainframe.

This data contains insights that will power their product and customer strategies for years to come. But getting to that data isn’t easy.

In this blog, we look at the challenges in the payments space and how you can use the power of the cloud to unlock your mainframe and become competitive again.

The State of the Public Cloud in the Enterprise: Contino Research Report 2020

The State of the Public Cloud in the Enterprise: Contino Research Report 2020

Why is the public cloud the greatest enabler in a generation?

We asked 250 IT decision-makers at enterprise companies about the state of the public cloud in their organisation.


Challenges and Threats in the Payments Ecosystem

Challenges for Payments Providers

Common Pain Points Across Payments Ecosystem

Despite all diverse challenges that exist across the different actors in the payments ecosystem (e.g. merchants, acquirers, processors, banks etc.) there are common pain points:

1. Losing customers

  • Fewer customers
  • Lower lifetime value

2. Losing relevance

  • Younger, more demanding market
  • Tech-savvy consumers

3. Losing market share

  • More competitors
  • Wider range of products

4. Losing revenue

  • Fewer payments
  • Lower fees

Threats to the Foundation of the Payments Market

Incumbents vs Emerging Competition

Payment Providers Need to Evolve

You need to shift your business from delivering resilient, reliable providing real-time integrated data-driven services.

But what’s the first step?

Payment providers are sitting on a GOLD MINE of customer data in their mainframes that they can leverage to regain competitive advantage.

But that data needs to be available for use by data science teams, your marketing team, your business analysts...all in real-time!

That batch export from your mainframe once a night doesn’t cut the mustard anymore.

And there are a whole set of hurdles stopping you from moving forward.

5 Reasons Why Payment Providers Can’t Unlock the Mainframe

1. Migrating the mainframe is risky and expensive

Mass migrating live data from old mainframes is very difficult and potentially costly. Both in terms of the migration itself and in the disruption to your business.

2. They can’t risk disrupting the customer experience

The data is highly sensitive and is constantly in use by customers.

Resilience is your USP, so if you disrupt the customer over.

3. Lack of digital experience

Cloud, DevOps, big data...this isn’t where payment providers have been concentrating their efforts!

4. Build for stability, not for agility

Payment providers have been focused on resilience. Now that circumstances call for change, they are struggling. They are built for stability, not agility!

5. Lack of time and vision

The focus is always on the immediate firefighting within an organisation, rarely looking up to consider where the industry is going and how to adapt.

How to Get Back in the Game: Liberating Your Mainframe

Liberating the data in your mainframe will unlock game-changing opportunities:

  • Data-driven product strategies
  • Deep customer understanding
  • Insights into industry trends
  • Optimise existing business operations
  • Predict and mitigate customer churn
  • Deliver automated personalised services

The companies that can unlock THIS are those that will dominate the next ten years.

But how to overcome the biggest hurdle? Getting to the data safely?

Our Solution: Live-Streaming Mainframe Data to the Cloud

How to avoid the risk, cost and disruption of mass migrations?

Real-time streaming solutions that deliver transaction data directly to the cloud. You can process and stream millions of transactions to the cloud, where they will then be available for analysis! It’s low risk and high value.

Our approach to unlocking the mainframe is to use a data-driven application pattern that streams transactions to the cloud environment.

This consists of four stages:

  1. Stream the data
  2. Use the data
  3. Store the data
  4. Exploit the data

Four data stages

1. Stream the Data


The first step is to replicate the information on the mainframe system into the cloud.


This is done in using cloud-native, real-time streaming platforms like Managed Kafka, AWS Kinesis Data Streams, GCP Pub/Sub or Azure Service Bus.

There are multiple ways of streaming the data based on either pre- or post-arrival triggers that we define depending on the client’s circumstances.


Your mainframe data is now instantly available in the cloud for consumption by other teams and services.

2. Use the Data


Make use of the incoming data in real-time to enhance existing services.

For example:

  • Real-time user profiles based on session behaviour
  • Real-time trade settlement dashboard
  • Real-time revenue accounting
  • Machine learning-based anomaly/fraud detection
  • Real-time data refinement and data pipeline
  • Enriching customer services in real-time
  • Providing personalised services
  • Answering queries based on real-time information


Use highly-scalable stream-based processing services in the cloud such as Apache Flink, AWS Lambda etc.

Your application teams can then input these data streams into existing services to enhance and personalise them.

It is here also that machine learning models (see part four) are deployed, using the real-time data stream to generate prediction and trend insights.


You can generate insights and predictions in real time that tell you which actions to take to generate business value.

3. Store the Data


Ensure the data is retained in a secure and flexible offline store so it can be used by your data scientists.


Utilise the flexibility of the cloud to meet your requirements with a Data Lake, Data Warehouse or Operational Data Store (ODS), depending on the exact use case.


Your data is now democratised. Data scientists can use it to train their machine learning models, do sophisticated analytics and provide insights to the business for data-driven product and marketing strategies.

4. Exploit the Data


Generate business value by providing data-driven evidence for the next-best action, whether that be across product development, marketing campaigns or customer service


There is a growing suite of cloud native machine learning and artificial intelligence tools and services that data science teams can start using instantly.

From interactive analytics and real-time machine learning to natural language processing and image recognition.

Data science and business intelligence teams can query and report on the data at scale using tools such as Jupyter Notebooks.

They can also enrich the data from the mainframe with existing data sets for even deeper insights.


This is the stage where data is turned into knowledge and then into actions. For example, knowing that a transaction might be fraudulent is knowledge that can generate an appropriate action.

Generating value-adding actions at scale and in real time is an absolute game-changer.

How to Rapidly Formulate New Products and Propositions

1. Bring Your Data Science Team in From the Beginning

While the platform is being prepared, bring your data science team on board to familiarise themselves with the technology, the analytics possibilities and the kind of data they will be able to run.

2. Establish an Exploration Squad

An Exploration Squad is a team dedicated to uncovering new value propositions. They derive insights, design new services and rapidly experiment to test market fit. At Contino, we use a six-week sprint to design, build and test new MVPs to validate potential new products and services.

3. Bring Together Your Teams for Maximum Value: Platform, Data and Product

Real business value emerges when you align three areas:

  • Platforms: Cloud, DevSecOps, SRE
  • Data: Stream Processing, Data Science & Machine Learning, Analytics
  • Product: product owners, customers, business analysts

These three areas working together enable faster insights, quicker design and rapid experimentation.

By accelerating your cloud, data AND product programs, you can start delivering business value from real-time transactions in the cloud in around 6-12 months.

What to Do After You Liberate Your Mainframe: Taking Home the Prize!

Once you have liberated the data from your mainframe, you can use it to drive a host of winning business opportunities:

Killer Products

Leverage masses of data to customise products and optimise offerings.

  • Transaction enrichments:
    • Real-time transparency to manage businesses
    • Up- and cross-selling via contextualised offerings
  • Expand into adjacent markets

Enhance Customer Experience (CX)

Real-time data feeds available to support CX.

  • Improved experience via payment ecosystems:
    • Digital onboarding experiences
      • Digital ID recognition
      • Digital Know-Your-Customer (KYC)
    • Customer-facing process automation
      • Dispute management chatbots
  • Personalisation
  • Security and fraud detection:
    • Real-time fraud analytics

Automate Insights

Generate data science insights using machine learning and artificial intelligence.

  • Churn prediction
  • Trends analysis and reporting (B2B and B2C)

Continuous Innovation

Leverage the agility of the cloud to innovate over the long-term.

  • Identify new revenue streams
  • Purchase or partner with FinTechs as part of customer journey

Real-Life Case Study: A Global Payment Processing Company

The client is Fortune 500 global payments processing company.

The Challenge

The client was losing market position to the likes of Stripe and Square who were bringing innovative new services to market much faster.

The task was to replace their legacy platform with an Enterprise Data Platform with machine learning at the core. This would enable real-time fraud detection services among other new services.

The Solution

  • Automated Platform Build: Use DevOps approaches in AWS to deploy a platform provisioning pipeline for the Enterprise Data Platform
  • Upskilling: Give the client’s teams the cloud and data skills to deliver and maintain the new platform

The Business Outcome

  • Rapid Deployment: the core platform deploys to the cloud in minutes rather than weeks
  • Cost Savings: infrastructure-as-code pipelines save £200k in terms of saved man-hours
  • Increased Margins: new fraud detection service reduces the drop-out rate for transactions, increasing per-transaction profit

Contino has helped countless enterprises on their transformation journeys. We have a tried-and-tested framework for transforming enterprise IT to a data- and digital-centric operating model.

Get in touch to find out how we can help!

More Articles

Consumer Data Right

The Role of the Consumer Data Right in Today’s World

12 May 2021 by Michael Ewald
Casey and Neil Contino Live

The Future of Tech Talent and Recruitment Is NOW

29 April 2021 by Neil Shmuely
CICD Pipeline

A Definitive Guide to Understanding the CI/CD Pipeline

26 April 2021 by Chris Murman