Explore a 38-minute conference talk from GOTO Chicago 2024 where Dave Thomas, founder and software industry expert, shares insights on efficient and reliable database migration. Learn why data migrations frequently fail due to complexity underestimation through a real-world case study of a major ERP company's large-scale migration that required high performance, reliability, and tight deadlines. Discover innovative techniques including bulk data transfer, table-driven approaches, functional prototyping, and specialized IV&V testing methods. The presentation covers the reality of data migration challenges, a critical legacy problem case study, reporting solutions, migration approaches, logical and physical models, binary and batch processing benefits, efficient serialization, risk-reducing implementation strategies, and comprehensive verification methods. Gain valuable strategies applicable to data engineers, architects, and development leaders facing migrations of any scale, with insights drawn from distributed systems research applied to legacy system modernization.
Efficient Reliable Database Migration: A Legacy Innovation Story
Overview
Syllabus
00:00 Intro
01:53 Reality of data migration
02:21 Case study: Critical legacy problem
05:38 Reporting problem
06:42 Reporting solution
07:45 Near-term solution
08:22 Long-term migration challenge
09:37 Conventional migration approach
10:52 Innovative solution
12:15 Logical & physical model
13:34 Essence of approach
15:37 Binary is better
16:09 Batch is better
16:43 Efficient serialization
17:49 Conservative implementation strategy to reduce risk
20:28 Verification & validation approach
29:53 Readers & writers?
32:53 Recap
37:39 Outro
Taught by
GOTO Conferences
