Posts Tagged "data migration"

Do you currently have a problem with migrating data from one system to another? Do you wish that the current manual method could be automated? Do your qualified staff spend time on the mundane activity of transferring data from one system to another?

If you answer yes to any of these questions, then Tata Technologies may have a tool available to solve these probelms – i Migrate It.

Picture2

This tool is an on-demand solution for any mannner of migrations and translations. It is configured for a given situation and allows for a user specify on demand what data he needs migrated and in what format.

From a user perspective, a typical workflow would be as follows:

  1. User logs into the tool. Configuration of the tool would determine what that specifc user is allowed to do (examine only, examine and migrate etc.)
  2. The user searches for the data that needs migration. Based on the search result, user chooses the exact data set required and specifies what must be preserved during the migration (full geometrical feature definition, brep only, metadata only etc.)
  3. Once all this is completed, the user would submit the job for processing. At this point, i Migrate It would take over and run the necessary background tasks required to complete the request. Depending on the nature of the systems, the job could take some time to complete (e.g. overnight batch process). The user has access to a dahboard that shows the status of the pending jobs and historical jobs.
  4. If the job fails (for example the requested data has already been migrated), the user is alerted with an error message which can be used to determine a future course of action.

This tool has several advantages for migration and translation problems:

  1. Only data that is really required by the users is migrated. This can reduce the cost of a complete migration.
  2. By providing options, the most efficent process is applied as determined by those who really know.
  3. After a period of time, usage will drop and the tool can be eventually phased out.
  4. Data remains secure during the process.

Consider this option when next you are faced with a migration problem!

Read Part 1 here.

So, what does a structured process to data migration and translation look like?

First a few definitions:

  • Source system – the origin of the data that needs to be translated or migrated. This could be a database or a directory structure.
  • Target system – the final destination for the data. On completion of the process, data in the target should be in the correct format.
  • Verify – Ensure that data placed in the target system is complete, accurate, and meets defined standards.
  • Staging area – an interim location where data is transformed, cleaned, or converted before being sent to the target.

The process consists of five steps as shown below:

picture1The process can be described as follows:

  • Data to be migrated is identified in the source system. This is an important step and ensures that only relevant data is moved. Junk data is left behind.
  • The identified data is extracted from the source system and placed in the staging area.
  • The data is then transformed into a format ready for the target system. Such transformation could be a CAD to CAD translation, a metadata change, or a cleaning process. Transformation may also entail data enrichment – for example, append additional properties to the objects so they can be better found in the target system.
  • Transformed data is then loaded into the target system. This can be done automatically via programs or manually, dependent on the chosen method. Automatic routines can fail and these are flagged for analysis and action.
  • Once data is loaded, validation is carried out to ensure that the migrated data is correct in the target system and not corrupted in some fashion.

The process as described above is shown at working level:

picture2

Shown in this diagram are two software tools – extractors and loaders. These are usually custom utilities that use APIs, or hooks into the source and target systems, to move the identified data. For example, an extractor tool may query a source PLM system for all released and frozen data that was released after a given date. Once this search is complete, the data identified by this will be downloaded by the extractor from the PLM system into the staging area.

In a similar manner, a loader will execute against a correct data set in the staging area and insert this into a target system, creating the required objects and adding the files.

It is highly recommended that pilot migrations be carried out on test data in developmental environments to verify the process. This testing will identify potential bugs and allow them to be fixed before actual data is touched.

Such a structured process will guarantee success!

What is data migration and translation? Here is a definition that will help:

  • Information exists in different formats and representations. For example, Egyptian hieroglyphics are a pictorial language (representation) inscribed on stone (format)
  • However, information is only useful to a consumer in a specific format and representation. So, Roman letters printed on paper may mean the same as an equivalent hieroglyphic text, but the latter could not be understood by a English reader.
  • Migration moves data between formats – such as stone to paper
  • Translation moves data between representations – hieroglyphics to roman letters

What must a migration and translation achieve?

  • The process preserves the accuracy of the information
  • The process is consistent

In the PLM world, the requirement for data translation and migration arises as the result of multiple conditions. Examples of these include changes in technology (one CAD system to another CAD system), upgrades to software (from one level of a PLM system to later version), combination of data from two different sources (CAD files on a directory system with files in a PDM), acquisitions and mergers between companies (combine product data) and integration between systems (connect PLM to ERP).

However, migrations and translations can be fraught with problems and require considerable effort. Here are some reasons: […]

© Tata Technologies 2009-2015. All rights reserved.