The Daily WTF
Follow
Floating Along
The author describes a data migration from one Microsoft platform to another, which encountered an unexpected issue. Initially, data extraction and loading were being handled by SSIS, but the author switched to PowerShell for more control. The goal was to migrate customer documents and metadata from SharePoint on-premises to a SQL Server database. A key field, the customer number, was stored as a Number type in SharePoint. Under the hood, SharePoint's Number fields are represented as Doubles, which is sufficient for accurate storage of ten-digit numbers. However, the SSIS-generated SQL Server schema used a 'float' data type for this field. The author incorrectly assumed that the '.Net' type 'float' in their PowerShell script would perfectly align with the SQL Server 'float' type. This mismatch occurred because SQL Server's 'float' defaults to double-precision, but the author didn't realize this difference in representation. Consequently, when numbers were transferred and converted, large customer numbers lost their last few digits due to precision loss. While testers initially missed the error, it was discovered during the full data load. The team had to identify the corrupted numbers and manually correct them after the migration.