Migrating from Microsoft SQL Server Compact to SQL Server: Best PracticesMigrating from Microsoft SQL Server Compact (SQL CE) to a full SQL Server instance is a common step when applications outgrow an embedded, file-based database. SQL CE is lightweight and easy to deploy, but it lacks advanced scalability, high-availability, and management features available in SQL Server. This guide covers planning, preparation, migration methods, testing, and post-migration steps to help you execute a reliable, low-risk migration.
Why migrate?
- Scalability: SQL Server supports much larger databases, better concurrency, and resource tuning.
- Manageability: Centralized administration, automated backups, and monitoring tools.
- Security & Compliance: Advanced security features (encryption, auditing, row-level security) and compliance options.
- High availability & performance: Clustering, Always On Availability Groups, and advanced indexing/partitioning.
- Integration: Better tooling for ETL, analytics, and integration with other Microsoft products and BI platforms.
1. Assessment and planning
Inventory and analyze
- Identify all applications that use SQL CE and catalog database files (.sdf), schema versions, and connection strings.
- Determine size, growth rate, and peak concurrency requirements.
- List database features in use (tables, indexes, constraints, relationships, stored procedures if any via application logic).
- Identify third-party libraries or frameworks dependent on SQL CE (e.g., older versions of Entity Framework).
Define target environment
- Choose target SQL Server edition (Express, Standard, Enterprise, Azure SQL Database, or Azure SQL Managed Instance) based on size, performance, HA, and budget.
- Decide on single instance vs. cloud-managed service. Consider Azure SQL if you want PaaS benefits.
- Plan server sizing: CPU, memory, storage IOPS, and network bandwidth based on current metrics and expected growth.
Choose migration approach
Common approaches:
- Script and deploy schema then bulk copy data.
- Use tools (SSMA, Data Migration Assistant, custom ETL).
- Use application-based migration (read from CE and write to SQL Server via code). Choose based on downtime tolerance, complexity, and available tooling.
2. Schema compatibility and conversion
Differences to address
- SQL CE supports a subset of T-SQL and lacks server-side programmable objects (stored procedures, functions) and advanced data types.
- SQL CE indexes and constraints behave slightly differently; composite index support exists but specifics may differ.
- Identity handling, datetime precision, and Unicode collations may vary.
- No support in SQL CE for views, triggers, user-defined types, or certain constraints — these may need redesigning.
Steps
- Extract schema from SQL CE. You can use tools or script generation to get CREATE TABLE statements.
- Adjust data types (e.g., SQL CE’s ntext -> nvarchar(max) replacement if used).
- Add or refine constraints, foreign keys, and indexes to match SQL Server semantics.
- Define proper collations and consider case sensitivity (SQL CE often uses case-insensitive collations by default).
- Implement missing server-side logic in T-SQL (stored procedures, triggers) as needed.
3. Data migration methods
Method A — SQL Server Migration Assistant (SSMA) for SQL Server
- SSMA supports migration from SQL CE to SQL Server; it can convert schema and migrate data.
- Pros: Automated schema conversion and data movement; logs and error reporting.
- Cons: May need manual fixes for complex schema or logic.
Steps:
- Install SSMA and its prerequisites.
- Create a project and connect to the SQL CE .sdf file and target SQL Server.
- Analyze, convert schema, review conversion report, then deploy schema to target.
- Migrate data and review any data conversion warnings.
Method B — Script schema + bulk copy (BCP/BULK INSERT / SqlBulkCopy)
- Export SQL CE schema, create equivalent tables on SQL Server, then perform bulk copy.
- Use SqlBulkCopy (C#) for programmatic high-speed transfers.
- Suitable when you want precise control and minimal tooling.
Example pattern (C# with SqlBulkCopy):
using (var sourceConn = new SqlCeConnection(sourceConnString)) using (var destConn = new SqlConnection(destConnString)) { sourceConn.Open(); destConn.Open(); var cmd = new SqlCeCommand("SELECT * FROM MyTable", sourceConn); var reader = cmd.ExecuteReader(); using (var bulk = new SqlBulkCopy(destConn)) { bulk.DestinationTableName = "dbo.MyTable"; bulk.WriteToServer(reader); } }
Method C — Application-driven migration
- If changing the application anyway, add migration logic to read rows from SQL CE and write to SQL Server, with validation, batching, and retry logic.
- Good for complex transformations or when you need to migrate gradually.
Method D — Export/import via CSV or intermediate files
- Export data to CSV and import using BULK INSERT or bcp. Useful when direct connections are not possible.
- Watch out for encoding, delimiter, nulls, and date formats.
4. Handling constraints, identity, and relationships
- Disable foreign keys and triggers during bulk load to improve performance, then re-enable and validate.
- Preserve identity values: use IDENTITY_INSERT ON when inserting explicit identity values.
- Order table loads to respect referential integrity (parents before children) or load without constraints then validate and enable constraints.
- For large datasets, consider partitioning strategy on SQL Server after migration.
5. Performance tuning during migration
- Batch inserts (e.g., 5k–50k rows per batch) to avoid long transactions and excessive logging.
- Use minimal logging where possible (simple recovery model during initial load) — switch back to full after completion if needed.
- Drop or delay index creation until after bulk load; create appropriate clustered index first if needed for performance.
- Monitor tempdb, log file growth, and I/O. Pre-size database files to avoid autogrowth stalls.
- Use multiple concurrent data transfer threads if network and server can handle parallelism.
6. Testing and validation
- Run data validation checks: row counts, checksums/hash comparisons per table, spot-check important rows.
- Verify schema fidelity: column nullability, defaults, constraints, and indexes.
- Functional testing: ensure application operations (CRUD, transactions) behave correctly against SQL Server.
- Performance testing: compare response times and concurrency characteristics; tune indexes and queries as needed.
- Regression testing: confirm business logic, reports, and integrations still function.
Validation examples:
- Row counts: SELECT COUNT(*) FROM Table;
- Checksums: compare HASHBYTES or checksum/CRC across matching primary key ordering.
- Spot-checks: sample rows by PK and compare field-by-field.
7. Cutover strategy and rollback planning
Options:
- Big bang cutover: downtime window where you stop the application, migrate final delta, switch connection strings, and bring the app up on SQL Server.
- Phased / dual-write: run both databases in parallel and incrementally move functionality; complex and requires synchronization logic.
- Synchronization tools: consider change-tracking replication, custom sync logic, or third-party sync tools for minimal downtime.
Rollback planning:
- Keep backups of original .sdf files and exported data snapshots.
- Maintain a fallback plan to switch connection strings back to SQL CE if critical failure occurs within the rollback window.
- Document migration steps and have scripts ready to reverse actions where feasible.
8. Post-migration tasks
- Update connection strings and configuration management to point to SQL Server; store credentials securely.
- Implement backups, maintenance plans (index rebuilds/reorganizations, integrity checks), and monitoring/alerts.
- Configure security: least-privilege SQL logins, role-based access, and enforce encryption if required.
- Review and adjust application code for differences in SQL dialect or behavior (e.g., concurrency control, transaction isolation).
- Train operations staff on SQL Server maintenance and monitoring tools.
9. Common pitfalls and tips
- Underestimating storage and IOPS needs — pre-size files and monitor during load.
- Ignoring collation differences — mismatched collations can break joins or comparisons.
- Not handling identity values, foreign keys, or triggers correctly can cause data integrity issues.
- Failing to test on a copy of production data — always rehearse the migration with realistic data and load.
- Forgetting to update connection strings in all deployment environments (dev/test/prod).
10. Tools and resources
- SQL Server Migration Assistant (SSMA) for SQL Server — automates schema and data migration from SQL CE.
- SqlBulkCopy (System.Data.SqlClient / Microsoft.Data.SqlClient) — high-speed .NET data transfer.
- BCP and BULK INSERT — command-line bulk import/export utilities.
- Data Migration Assistant (DMA) — for assessing compatibility and feature parity.
- Monitoring tools: SQL Server Management Studio (SSMS), Azure portal (for Azure SQL), and third-party APMs.
Checklist (Concise)
- Inventory .sdf files, app dependencies, and schema/features in use.
- Choose target SQL Server edition/environment.
- Convert/adjust schema and data types; script tables and indexes.
- Select migration method (SSMA, SqlBulkCopy, CSV, app-driven).
- Migrate data with batching, preserve identities, disable constraints during load.
- Validate data, run functional and performance tests.
- Plan cutover with rollback procedures and backups.
- Update connection strings, implement maintenance, monitoring, and security.
Migrating from SQL Server Compact to SQL Server requires planning and testing, but following these best practices will reduce risk and give your application the scalability and manageability benefits of a full SQL Server environment.