Skip to main content
Download free report
SoftBlues
Back to Projects

Cloud Data Platform for Neobank

Senior Data Engineer2021 - present timeDmytro K.
DK
Dmytro K.

Data Engineering Lead

Data Engineer & Big Data

Key Expertise

Cloud Data PlatformsData Engineering LeadershipData Warehouse MigrationFinTech & Banking

Experience

10+ years

Timezone

CET (UTC +1)

Skills

Languages

JavaPythonScala

Databases

Google BigQueryMongoDBOracle DBkdb+KineticaNeo4j

Infrastructure

GCPDataprocDockerKubernetesGrafanaKafka

Frameworks

Apache SparkApache Airflowdbt

Integrations & Protocols

Apache Atlas
7-day risk-free trial
Response within 24 hours
View Full Profile

Overview

End-to-end cloud data platform for a fast-growing neobank, with core event tables ingesting large volumes of data daily and hundreds of terabytes of cumulative data. The platform underpins reconciliation, partner integrations, and financial reporting – operating on a cloud-managed infrastructure where both reliability and regulatory compliance are non-negotiable. Given the nature of the business, any pipeline failure or data discrepancy carries direct financial and audit consequences.

Achievements

Automated reconciliation pipelines for primary payment gateways, eliminating the vast majority of manual effort and materially strengthening audit readiness. Built internal tooling for workflow generation that became a team-wide standard, significantly compressing the time from requirement to deployed pipeline. Improved execution times on core legacy workflows through architectural refactoring. Designed a data compaction system for cold storage that reduced both read latency and long-term maintenance costs. Identified and resolved a query efficiency issue that measurably reduced cloud compute consumption at the organizational account level — one of the more impactful cost engineering wins on the platform.

Responsibilities

  • Migrated reconciliation and partner-integration pipelines from self-hosted to cloud-managed orchestration, eliminating a category of infrastructure failure and freeing the team from operational toil.
  • Designed automated reconciliation pipelines for primary payment gateways, replacing error-prone manual processes with auditable, testable data flows.
  • Built internal tooling for DAG generation and operational workflows, establishing consistent patterns that reduced both development time and onboarding friction across the team.
  • Refactored legacy workflows using modular DAG patterns and dynamic task mapping, achieving substantial gains in average execution time.
  • Worked closely with finance and compliance teams to ensure data automation aligned with regulatory requirements.

Technologies Used

PythonScalaApache AirflowGCP BigQueryDataprocMongoDBKafkadbtDockerKubernetesGrafana
DK

This project was delivered by

Dmytro K.

View Full Profile

Ready to Build Your AI Team?

Get matched with the right AI experts for your project. Book a free discovery call to discuss your requirements.

We respond within 24 hours.