Production Concept Changes SOP

This document outlines the decision points and process for updating concept definitions within the Special Olympics code base

Background

The VecnaCares team has received requests to update concept definitions, even where data has been already entered. The purpose of these guidelines are to help decision making for such changes.

The changes have been grouped into two major categories:

1. Change of definition within the same data type

2. Change of data type

Change of definition within the same data type

The data type of the concept is not changed but some of the definitions are changed

Data Type	Change Description	Data Validation	Action	Remarks
Numeric	Change in critical high, normal high, normal low, critical low	None	None	These are usually for display purposes
Numeric	Change in absolute high, critical high, normal high, normal low, critical low, absolute low	Run query on existing data to ensure that the new limits are not exceeded	Report on data which falls outside the new limits
Numeric	Allow Decimal	None	None	None
Numeric	Remove Decimal	Determine what the rounding approach is so that the data is migrated	Liquibase changeset to round the data according to the selected approach	Rounding options are : round half up, round half down, Round half towards zero, Round half away from zero, Round half to even, Round half to odd
Coded	Add options	None	None	None
Coded	Remove options	Determine what happens to the data with the removed coded options - ignore, void it or move it to another option	Liquibase changeset to effect the data changes	The impacts of ignoring data need to be assessed as it may cause computation errors for certain metrics
Text	None	None	None	Added for completeness
N/A	None	None	None	Added for completness
Document	None	None	None	Added for completeness
Date	None	None	None	Added for completeness
Time	None	None	None	Added for completeness
Datetime	None	None	None	Added for completeness
Boolean	None	None	None	Added for completeness
Rule	None	None	None	Added for completeness
Structured Numeric	None	None	None	Added for completeness
Complex	Change Handler			To be determined based on need

Change of data type

This section handles data type changes. The overall process for such a change is as follows:

1. Create a new concept with the new definition

2. Remove the concept mapping from the old concept to the new definition

3. Run a data validation query to identify any issues that will happen during the data migration. The special cases for the migration issues are detailed in the table below to assist the identification of potential data loss

   • void existing obs record with void_reason "Datatype change"

   • create a new obs record with transformed data

4. Write a liquibase changeset for the data transformation from the old data type to the new one

Initial Datatype	New Datatype	Data Loss Validation	Remarks
Numeric	Coded	Identify numeric values which do not match the coded values
Text	Coded	Identify text values which do not match the available coded values
Coded	Numeric	identify coded values which do not match the numeric values
Coded	Date	identify the algorithm for conversion of the coded values to dates
Date Numeric Time Datetime	Numeric	determine the algorithm of conversion from the date to numeric values
Date Numeric Time Datetime	Coded	Determine the algorithm of conversion from dates to coded values
Date Numeric Time Datetime	Coded	Determine the date values which do not match the coded after conversion