Home Forums Copy imported items folder

When I backup AC, I exclude imported items.
I have been manually copying the II folder locally to another drive. (Don't worry, Bert, I have taken your advice to heart, and I have redundancy since I also back it up automatically to a "cloud" service).

For the local backup, is there a simple way to automate the copying of this folder? (without using backup software, just a simple copy)

Donna,
We use the XCopy command. Make a new text file with the following text on one line:

xcopy c:\program files\amazing charts\importeditems\*.* [[location where you want backup files to go]] /C /D /E /F /H /I /R /Y

Save the file as a .bat extension. Whenever you open this .bat file, it will copy all files in the imported items folder to the location you specify for backup. The next time you open this .bat file, it will copy only new or changed files to the backup folder. However, if you delete any files from imported items, the same file in the backup folder will not be deleted.

In [[location where you want backup files to go]] you can enter something like "B:\AC backup\" where drive B is your external HD or wherever you save your backups.

The commands /C /D /E /F /H /I /R /Y tells it to ignore errors and just keep copying.

It's not the perfect solution but it works well enough for us.

What is the purpose of backing the II up separately to the drive, instead of letting AC do it by selecting on the check box? Is the ENC file to large?

Yes they can get too large for reliable online backup.

We had regular problems with backups that were 2-3 Gigabytes. When we had Guardian Angel Support help us they had us turn off imports & pics. Online backups are now 99% plus reliable. Our ImportedItems folder is now 16.5 GB and our .enc backup file is about 1/3 of a GB. The longer timeframe the inline backup takes the more likely it will have some internet problem. Also, we found our cable ISP was cutting off any process running at midnight so we schedule them to be done before then or start after.

Oh, that makes sense. I don't do the online backup. (Think how long the restore will take.) I have a G-Safe 2 drive Raid box and swap out a third drive to take home now and then. So there is a hard disk backup, 2 onsite RAID images, and a 3rd image at home from a few days ago if the building actually burns down, everything stolen, etc.

... I hope BERT is NOT reading this....

Serene,
Thank you, that sounds like what I wanted.

I vaguely remember batch files from the good old DOS days when we used one to backup the first version of our practice management program.

Yes, I remember .bat files from those days too!

Question: if you backup imported items separately, what happens to the comments that normally get saved with them?! These are most important from a M-L standpoint. i.e. "I told patient to follow up on this abn pap smear and referred."

That is a good question. I have assumed that the comments are stored in the AC database somewhere.

I previously asked about whether there was a way to print a log of these comments other than accessing them one at a time with each item. No one replied, so maybe we don't know how it works. Perhaps we have to ask the Angels.

So, since I have been unsure where the comments are in the permanent record, I don't use the comments, but actually create a message for any important actions generated by review of an item. That way, I have a record in the prior encounters.

AC backup should have that option to generate one backup with imported items and one without imported items/create an incremental backup solution. Wow 16GB of imported items.

AC does give you the option to back up either just the .enc file OR .enc and imported items. It does not offer an incremental backup. We use Macrium reflect for that.

It would be pretty cool if it did. Let me see what I can do.

Yes, an incremental copy is what I wanted. Not a big deal at present, but it will be as the II folder gets bigger over the years. (I've gotta watch those file sizes, right JBS? )

I have Acronis and I could use it to backup the II folder, but I like the simplicity of a simple copy of the II file that I can drop into any computer without using a software restore routine. I find Acronis hard to understand although I know it is well regarded.

I have been using AC in FP office for over 4 years. We have over 30GB of data in Imported Items. It takes 3 days to upload to AC, so I back up AC without Imported Items then I use Backup Assist to store an image of the imported items folder. It runs this automatically every night.

First, are we talking about click the batch file and not opening it? I only say that, because it would be confusing.

Second, Acronis will allow you to grab the Imported Items. That' the beauty of Acronis. It is an image that you can mount and it is identical. Acronis used to be elegant, simple and just worked. I stopped using it when they started coming out with this and that...and backup and restore v500. Was so difficult, it wasn't worth it. Vaults...etc.

Third, backup AC, do what you want with II....but just back up your whole computer..then everything is backed up.

30GB?! That would take 7-8 days on a standard DSL line. I'd probably just keep a folder synced on an external drive. Services like Carbonite/Backblaze have incremental backup and my database in the normal ENC file to be backed up.

Hi Sandeep, I keep incrementals of the Imported Items folder on my server and on a separate location.

Are the backups in both locations incremental? Or is one backup incremental and the other backup copies 30GB every night?

Online use deduplication.

We use FAP instead and only II only has things like our lab reports and other in office created things that end up there.... Keeps 'em separate but useable together in the best of ways.... Although it does take a bit of time to back-up both of them each night... But AC is much longer than FAP even with those pesky docs being mostly in FAP....

it runs one incremental which is kept in a storage partition on the main server, and that incremental file is copied to an external drive. Do you recommend a different strategy?

Shouldn't you have a copy online (or at least, physically out of the office)?

For those of you who use JungleDisk, maybe you know this, but it is new to me today. As I continued to mull over the best way to copy my imported items folder automatically, I came up with this. I am using JungleDisk Desktop edition for my end of day online backup. In addition to online backup, JD provides an added disk, "J", network drive, that looks like a local drive on my computer even though it is online. I set up the AC II folder as a sync folder on this drive. You can then use JungleDisk on other computers to sync to it as well. You can set the sync interval to whatever you want. So, the II are synced to multiple locations throughtout the day rather than waiting for my end of day backup. Pretty nifty, like a continuous incremental backup! A nice use for those of us using P2P, not server/RAID setups. And the network drive can be accessed from anywhere with an internet connection. Perhaps old hat to you techies, but a discovery for me!

One of the problems I face is that my office has a DSL connection, and I have been concerned that as the II grew that backups online would be time consuming. I think that by doing it this way, it will solve that problem. The II folder on the J drive is also synced to a home computer, so if the office computer fails, I can just copy the file from my home machine and carry it back to the office, not having to retrieve it all online.

Wait...does this mean you are no longer worried about the size of your II??? This is big news!
Seriously, I would like to see how this works.

Nope, I am still concerned about size. The larger the files the longer the sync will take. But, it is better since it is being done in small parts rather than one group at the end of day.

My II file is 174 MB. It took 6 hours to sync from the office initially. At home, with the cable connection, it only took 1 hour. Speed at the office is definitely limiting.

I'd be happy to demonstrate it for you.

http://amazingcharts.com/ub/ubbthreads.php/topics/41080/Copy_imported_items_folder#Post41073

I was just talking about a direct link to the post as above. Not always easy to find the other thread.


Are you referring time backing up, restoring or both?

Both.

6 hours for 174MB? Donna, I think it is time to stop using AOL dial-up for this. You could probably print a copy of each document and take them home faster than that.

Not dial up.
This is a DSL connection.
See why I think about file sizes?

Donna,

I think he was kidding about dialup.

What version or type of Jungle Disk are you using? Workstation? Server.

Even if you had 500GB of data, once it is up in the cloud, the following day's backup should take around 15 minutes. JD uses De-Duplication, which is MUCH faster than incremental and certainly faster than differential.

There are cloud services which will deliver your data overnight using hard drives given it would take many hours to download a lot of data. So, it depends on why you are using it. If you are looking at an online site as a "mission critical, have to restore in the next hour" type of thing, it isn't. Most can't. I look at it as one more offsite backup repository. Say the building does burn down. My guess is you can download from JD faster than you can build a new building.

First thing, though, what type of JD are you using?

I am using the JD Desktop edition.
The daily backups have been taking from 2 to 40 minutes.

Is is the initial sync function to the "network" J drive that took 6 hours. This is a simple copy routine, not their backup process. I still think that this may work well for me as I have set it to sync every hour, so only small amounts of data uploaded at a time. I consider the JD files my very last resort. I will always use my local backups first.

That makes a lot more sense. But if that is the case, why worry about a big II? You only have to do the initial upload once.

The interesting thing about the way Jungle Disc works is that unchanged files are updated (or perhaps, verified is a better word) very fast.
So my II of 32GB took less than 1.5 minutes to back up today (a Sunday, so no files changed and the upload is 13KB).
My enc folder has 18 enc files in it, totaling 3GB; only one was changed (the new one created today) and that back up (of a much smaller file) took 4.5 minutes, uploading 136MB.

One take home point is that you could backup your II very frequently, very quickly, if you chose to, but backing up the enc's this way will be more of a problem and more time consuming.

Exactly. The first upload is what takes all the time. Donna, I ask about the version, wondering if you have updated to v3. I use the Server Edition, which already has deduplication, but I don't think v1 and v2 of Workgroup Edition did. Jon makes reference to it, but it is much more efficient than that albeit somewhat complicated at the ground level.

v3 now deploys block-level de-duplication with compression which allows the data take up a very small space. De-duplication or Single Instance Store will only upload changed files, but more than that will only back up changed blocks within the file. So, if you have a PHI form that you save to your computer, only the block that contains the new data (say the patient demographics) will be backed up and will be pointed to the matching data. This pointer is then placed in an index. This will keep you from having any redundant data. If you backed up 50 emails with the same 50 email attachments, only one of the attachments would be backed up. This allows small backups and very fast. The de-duplication process can be either server side or cloud side, both with advantages and disadvantages.

Files such as .enc backups are backed up in entirety as it is not possible to back up blocks of a backup file (that I am aware of).

This would lead to my question to Jon as to why have 18 AC backup files. Having a year's worth of computer data makes sense as you can restore individual files and folders, but with the .enc backup, it is all or nothing. It would seem that anything over five .enc backups would have a lot of lost data from AC. Just an observation.

I only did it because I wanted to see if I could get you to say "you are backing up too much".

I think the answer is not one that I can defend well, nor is it one that I feel strongly about. I suppose I could turn it around and say "why do you need 5?" (or 10, or 20)?

I may be totally off base about this, but I guess the idea is that the enc file could be corrupted. If so, you might not discover this until a day (or 3,5, or 10) later. In that case, you would want to go back to the last file that was still healthy. I know...this is all pretty unlikely, but as we all know, unlikely scenarios is what complex backup regimens are for.

Yes, I am using V3 of JD.

The backup that I had been running included my AC backup folder and the II. Even with the small II, some days it took up to 40 minutes. So, I was trying to project down the road. If I do need the data back from JD, it would be very time consuming. I have not seen that they offer an option to return the data by mail.

Thanks for all the replies; it has been educational and helped me to think through my plans.

Donna,

I back up over 300GBs of data to JD. (Server Edition) It did take quite a long time. After this, it does take quite less time. Jon did hit on something. I wonder if the de-duplication process struggles a bit with certain files such as backup files.

My server backup at night takes over two hours. So, depending on how many other backups you are doing, 40 minutes isn't all that long.

Every once in a while, say quarterly, I back up my data partition to a new vault and delete the other. This brings the backup times dramatically lower.

I would be interested in how many AC .enc backups you have in your AC backup folder. Personally, I find more than five to be excessive for many reasons. Keep track of the time it takes every time you add another backup.

If I may offer once piece of advice or philosophy. I am guessing you keep local backups of your AC backup folder and imported items. No, JD will not send you a backup hard drive. But, I do not think it is prudent to consider any online backup as your immediate restore. I consider my online backup to be the equivalent of taking a copy home. Why do we keep an online backup or offsite backup? In case of a disaster such as a fire or flood or hurricane. That would be the only reason. If your office were devastated and all local data was destroyed, you would have ample time to download the files to a computer.

Bert,
The longest recent backup was 4 days ago, uploaded 40 MB, took 44 minutes. There are 22 enc files in the backup folder. I usually clean it out monthly, perhaps I should do it more often. Today's backup took less than 2 minutes.

And, yes I have local copies. JD is my last resort. If Baltimore is wiped out, but my patients and I survive somehow, I can get my data from JD to go on.

Thank you, I do feel that I have a good plan now, so I shall sleep soundly.

I have learned from this, too. Your II file could grow in size by an order of magnitude, but the time to do an incremental back-up (I know, actually a de-duplication back-up) might only increase by 10%.

On the other hand, each day's new enc file requires a much longer time to back-up due to the larger "block" of data. It seems that the NUMBER of enc files that you are saving is pretty much irrelevant. If you have saved day one, on day two the upload is JUST the day two file (day one is unchanged). By day 20, your first 19 are unchanged so the upload is JUST the day 20 file (days 1-19 are unchanged).

***The combination means that over time, you can back up both enc and II, and expect minimal increase in back-up time, even with substantial growth in your file size and number.

And Donna, you say "If Baltimore is wiped out, but my patients and I survive somehow, I can get my data from JD to go on." Bert's point is in that scenario, you may want to retrieve your JD data, but it won't matter if it takes some hours to do so.

Yes, That's what I was alluding to. In the case of a catastrophe, just the fact that I would have data is important, no matter how long it takes to get the data back.

I really like the sync function of the JD network drive. So, today's II were uploaded throughout the day. When I got home and turned on the computer, JD synced the folder at home within a couple of minutes. For me, this is a good solution for the II, totally automatic, and I end up with an online copy and a local offsite copy.

OK, say you have this hypothetical scenerio: Your office is burglarized and they only take your main computer and your local backup. So you have to goto your online backup. Say you use jungle disk. You have your *.enc file backed up with them but it does not include imported items. You have the Imported item folder backed up with jungle disk also. So you want to get up and going quickly. You download the *.enc file and restore to another computer and it works perfectly. You start to download the 16 GB imported items backup. My question is can you go ahead and start using Amazing Charts before you get the Imported items back? I know there is an option in the utilities to reindex imported items. If you finally get the imported items back a couple of days later are there going to be sync problems? Can you just drag the Imported items folder into the new amazing charts imported items folder?