Test data management

1. Understand

1.1. Adequate test data is available to run full automated test suites

2. Pitfalls

2.1. Sensitive test data is not anonymized / hashed / offuscated

2.2. Test data is obsolete - no longer relevant

2.3. Over reliance on data for testing, specially for unit testing

2.4. Over reliance on DB access or external data leading to

2.4.1. slowness

2.4.2. costs

2.4.3. Linit the test the team can run

2.5. Requireing full copy of production DB instead of portion

3. Implement

3.1. Favor unit test

3.2. Unit test should not rely on external data

3.3. Used test data only when needed to lower its maintenance / relevance cost

3.4. Isolate test data to enable hermetic testing

3.5. Minimize test data in databases

3.5.1. Poor test isolation

3.5.1.1. Need clean up for idempotency

3.5.1.2. difficult to parallelize, mean slow

3.5.2. Performance impact

3.5.2.1. DB read / write perf < local data

3.6. Identify portion of meaning full up-to-date full data to export instead of slow full export

3.7. e,g, Use PubSub snapshot and message relayping feature to test subcriber code on known data

4. Measure

4.1. % time used to manage missing / obsolete test data

4.2. % of key data set that are available

4.3. % of test that can be run without to aquire new test data