How to copy data only from production to development

[comma]

Hi guys,
If there are some data changes on my production model, and I want to copy this to the development, do I only have to copy the .cub files?
If yes, which .cub files? Only the user defined cubes or including the system defined?

Sorry if the question sounds dumb. Thanks.

[Alan Kirk]

Hi guys,
If there are some data changes on my production model, and I want to copy this to the development, do I only have to copy the .cub files?
If yes, which .cub files? Only the user defined cubes or including the system defined?

Sorry if the question sounds dumb. Thanks.

No, to be safe you'd need to copy at a minimum:
(a) The .cub file for the cube that you're copying to development;
(b) The .dim files for each of the dimensions relating to that cube;
(c) The }ElementAttributes_ .dims and .cubs for each of those dimensions.

Whether you copy the view and subset files is up to you. However if you don't take the .dims as well then if new elements have been added in production but they aren't there in the development version of the .dims, you'll lose the corresponding data when you load the cube in your Dev environment.

You'll probably also want to take the .rux file if any to ensure that you have the correct set of rules.

Finally, you may want to take any .xdi or .xru files if you're using those. (Though you probably won't need them unless you're planning to test new rules or hierarchies as well.)

[paulsimon]

Hi

I am guessing that you have a scenario where you have made changes to rules, dimension hierarchies, views, etc, in your development environment and you want to get the latest production data back for test purposes. I am guessing that you don't want to just take a full copy of production otherwise that would over-write the changes that you have made in dev.

There are a couple of ways to do this:

1) If your data load routines are or can be set up to do a full load of production then you can point your data load routines at the production source data and load from there.

2) If (1) is not possible, eg due to the volume of historic data, or due to data being entered, or due to security restrictions, then the other option is to export all data from each cube in production to csv files, get these copied back, and then load these in to the cubes in Dev. I have TI processes that can do this. Essentially it is just a loop for each cube and
}ElementAttributes cube to export the data, and then a similar loop to import the data. My routines cope with cubes that have up to 20 dimensions but could be extended to cope with more dimensions if necessary.

3) You might get away copying the .cub files only, as later versions of TM1 are a lot better at re-synching dimensions with cubes. However, option (2) is probably the more acceptable approach.

Regards


Paul Simon

[Alan Kirk]

3) You might get away copying the .cub files only, as later versions of TM1 are a lot better at re-synching dimensions with cubes. However, option (2) is probably the more acceptable approach.

Could you clarify that one? Are you working under the assumption that the OP has synchronisation happening from Production to Development, or are you suggesting that a copied cube file can "grow a new limb" by regenerating missing elements in dimensions? Because the first seems unlikely, and the second isn't something that I've heard of and would be potentially dangerous if it existed. (It also doesn't match with a test that I did in 9.5.1; added an element, wrote data to it, saved the cube, stopped the server, took a copy of the cube, restarted the server, deleted the element, shut down the server, copied back the original cube, restarted the server, re-added the deleted element, opened the cube view and... presto, no data.)

You need the .dims because, unless something has changed in recent versions, you will lose data without them. The trade off for that is that if you've been monkeying around with the hierarchies in dev, then yes, you'll lose those. But it's a choice of one or the other, not both. The rules are another matter which is why I said that the rules file was optional.

[tomok]

(It also doesn't match with a test that I did in 9.5.1; added an element, wrote data to it, saved the cube, stopped the server, took a copy of the cube, restarted the server, deleted the element, shut down the server, copied back the original cube, restarted the server, re-added the deleted element, opened the cube view and... presto, no data.)

This is not new behavior in 9.5.1, this is what TM1 has done for as long back as I can remember. However, if you would have stopped the service, without saving, immediately after adding back the element and then restarted the service, your data would have magically reappeared. The key is you can't save the .cub file. Once that file is re-written your data is lost. As long as you have the version of the .cub file with the data in it you will get the data back after you 1) add back any missing dimension elements and 2) recycle the service.

[Alan Kirk]

(It also doesn't match with a test that I did in 9.5.1; added an element, wrote data to it, saved the cube, stopped the server, took a copy of the cube, restarted the server, deleted the element, shut down the server, copied back the original cube, restarted the server, re-added the deleted element, opened the cube view and... presto, no data.)

This is not new behavior in 9.5.1, this is what TM1 has done for as long back as I can remember.

Yes, that's what I was saying; to the best of my knowledge it hadn't changed and, based on that test it doesn't seem to have. I'm unclear on what Paul was referring to.

However, if you would have stopped the service, without saving, immediately after adding back the element and then restarted the service, your data would have magically reappeared. The key is you can't save the .cub file. Once that file is re-written your data is lost. As long as you have the version of the .cub file with the data in it you will get the data back after you 1) add back any missing dimension elements and 2) recycle the service.

Though you're talking about adding back the element in the same session as it was deleted. I'll take your word for that; I'm not going to test that one out because it's not germane to the issue in question, that of copying the .cub to a different environment, then adding the elements back in. (Aside from which when you shut down a service then by default the server tries to save anyway; you'd have to knock it on the head from Windows to stop that happening.) Essentially the latter test is what I did by using the copy of the .cub file which had been saved with the data before the element had been deleted.