Home Forums Discussions Problems V5 has Critical Bug - frequent drop at SQL server

We are using Windows Server 2003 for our SQL data with about 30 computer terminal. 8 computers are using Terminal Server and the other 22 are P2P with the server not using a Domain. We have frequent problems with connection to the data. The terminals were dropped in the middle of seeing a patient every day. We have turned off the firewalls. After going thru the diagnostic process, the error message in the Virtual IT is: " I couldn't stop the SQL Server Service. Frequently this is because Windows is in the middle of something and didn't respond in the time allotted. So please click the step 3 again and give it some more time."
We have learn the trick to connect again, but the problem continues every day. We are going to contact the AC support and see whether they can look at the error log inside our SQL server data base.

Do any of the terminal server connect instances of AC drop their connection? For instance, if you drop on one of the P2P machines can you still use AC from a Remote Desktop connection?

Philip, I have seen this issue in our office also, but then I was running beta version of AC. I am hoping it wont happen after installing the full version.

We have 5 PC's and 3 dumb terminals connecting using TS. I also dont have a domain.

I can find almost no log entries in sql or server 2008 event log that gives me a clue why the connections drops

Thank you for your suggestions. Eric from AC tech support has been trying to fix the problem in the past one week with no luck. He uploaded and checked our SQL Amazingcharts primary data file(around 800MB) and condense the transaction log file from 300 down to 120MB. We reinstall the whole program in the server, problem persists. It happens randomly in both terminal server and the P2P clients. It seems that the total number of computers logging into the server and the TS client number approaches 28-30, the problem will comes up. We are sure that this is not our network or Windows license issues because we can access the other scanned data file in this server with no problem.
It really makes me wonder whether the SQL Express actually has a limit for the number of clients logging in at one time. Our IT said that there is also a 1GB data limit on the SQL Express and some how our primary data file increased from 600MB to 800MB after upgrading from V4 to V5. We might be forced to switch to other EMR when the data size is approaching 1GB. We really need some help!!

The documentation says that the database size limit is 4 GB. However, SQL Express will only utilize 1 GB of RAM (regardless of the installed RAM on the server) and 1 CPU, even on multiple CPU machines, so perhaps this is the reason that, when a large number of machines are logged in, connection failures occur.

Hi John,
Thank you for your clarification. Do you think put the Amazingcharts program and data in a solid state hard drive can bring up the speed and prevent the frequent drops?

Before you invest in any hardware, consider posting a question on Microsoft's TechNet board for SQL Server Express . I already did a keyword search for dropped connection, connection timeout, and maximum clients, but nothing useful.

Bert may want to chime in here; I know he has a favorite server/networking board that he has recommended in previous posts.

John, you read my mind. I thought I would weigh in. As I am nowhere near a SQL guru other than being able to spell it, I did go to my favorite site for some help from the SQL experts there. Rather than pick out certain ideas or make them appear they are from me, I have permission to copy and paste it directly: (this is a post from a user there)


From Bert: I also wonder about the number of AC clients being used and would love to know how many other practices have that many users. Did support comment on that? Certainly not trying to scare you aware from AC. If you have the memory on the server, it will definitely take it. There really are no "memory leaks" which are famously attributed to SQL, it's just that it will take what it can get. So, with 4GB, really = ~ 3.2GBs, you will definitely get more.

I agree that part or all of this may be your overall network. I would tend to want to start from the router to the switch to the server to the clients and make sure everything is 1GB or whatever you are set up for. You could also consider upgrading the server to 2008 or even SBS 2008. SBS would give you the ability to have 75 users and would have the advantage of having SQL Server 2005 already installed. And, it is 64 bit, which will allow you to add tons of memory depending on the motherboard. My guess is you are running 32-bit on Windows Server 2003 and could go to 64 bit if your hardware supports it, but if you do upgrade, you may as well go to 2008 and, if money is not an issue, consider SQL 2008.

You could also simplify things and try taking certain things off the network such as just not using or disconnecting the terminal server ones. Just curious, why are you not using a domain. My guess is it is because with that many computers, some don't have the correct OS?

That wouldn't be AC's issue, it would be SQL Express. (although it is 4GBs of data and 1GB of RAM as mentioned above) You do not HAVE to run AC with Express, in fact I don't. I also have 20GBs of RAM which is an entirely different story.

I have seen a diagnostic tool used to quickly identify root cause of problems like this. It is called BMC Appsight. The AC support team would be in the best position to do advanced troubleshooting with a tool like this as they could develop and then compare against a set of baseline transactions.

Eric,

I took a look at the site. I am always wary of sites that don't advertise prices. I could have missed it. They mention that they saved Motorola $11,000,000. That doesn't sound like the type of software you can download for $49.95. Do you know the cost?

It looks rather impressive, though.

I have a call in to BMC for current pricing arrangements. It was very expensive (think 3 zeros+ vs 2) when we bought licenses in my last job but this was before BMC acquired the company. Appsight was geared to software development companies, large corporations and value added resellers like EDS, IBM or HP services (hence my statement about AC being logical point for using this tool vs. an individual office)

Oh, I missed that part. Makes more sense. Yes, AC investing in it may make sense. Thanks for the update, Eric.

Philip,

Did you have any luck. Any feedback would be appreciated.

Hi Bert,

I am sorry that I cannot get back to you earlier. We had multiple incidents of dropping from the SQL Exp yesterday. AC tech support logged in to our system again and clean up some old Access files in the server. He said that he will bring this issue to the developer.

You mention about checking the whole network. We have 2 un-managed switch, one Gigabyte switch connecting to the main server (Windows 2003), a back up server (Windows 2000)and majority of the laptops (Windows XP Home). Another 100MB switch connecting to printers and some other tower running AC. All except computers have 1GB RAM except one Dell Inspiron 1000 that can only take 500MB RAM. There are 3 older Dell Inspiron 1200 upgraded to 1GB RAM. They can run the AC program, but too slow to do the AC e-prescription. Do you think these 4 Laptops actually slow down the whole network?

The above network is connected via a VPN tunnel (using Sonicwall) to a second office with 9 computers and a second VPN tunnel to our billing company with 2 computers running AC. This set up was running fine using AC V4. The problem happens after upgrading to V5.024.

I really appreciate your input. I have contacted my IT concerning your suggestions. He is looking into a possible trial of the SQL server 2005. Another possibility is to remove all the computers from the 9 exam rooms, and let the doctors and nurses use wireless 4-6 netbook/Tablet at one time. However, I worry more about the stability of the wireless network.

Not all that complicated but there is a lot of different components working with each other. I am not sure I would run out and get new computers bases on the above. It's interesting that it happened when going from v4 to v5, which would make it fairly obvious it's some type of connection issue. This is over my head, so I hate making suggestions, although moving to SQL Workgroup couldn't hurt.

I still think you need to isolate and trouble shoot. Get a certain group of similar computers working without dropping SQL, then add on.

I know the difficult part for AC tech and it is also a frustration for them is it can be difficult for them to troubleshoot the issue when it is more of a networking issue than a v5 issue

Ultimately, you may need the services of someone who specializes in SQL Server databases and networking. There are many online and many who will provide remote service. I will try to get my SQL friend on Experts Exchange to take a peek in here.

Hi Bert,

Thank you for the advises. I may try to use the slower computer as TS client when every body is back to the office. Further help is turely appreciated.

Philip

Still trying. It's a complicated issue. I posted back over at Experts Exchange.

Today is the most frustrating day I ever have in the 6 years history of using AC. 4 out of 7 physicians are off for the Spring break. Work flow is slow. Yet AC kept dropping from the SQL server. No more than 20 computers could log in at a time. My office manager and I spent 3 hours with AC tech support on line and on the phone. We still cannot solve the problem. We just learned that we are not the only group facing this issue. I have to confess that I am a whole-heart supporter of the low cost EMR philosophy of AC. Yet, I cannot convince my other associates to bear with the problem any longer.

Philip,

I wouldn't give up. I am confused. You say you have been using AC for six years. Yet, this problem seems like it is rather new. From what I gather, your network is a little unique or at least a bit complicated. Did you try one of my suggestions, which is to simplify everything and start with one or two computers.

Anyway, as the question on Experts-Exchange has not been able to bear any fruit, I have sent an email to one of the experts there. This expert is a SQL Server 2005 and 2008 expert. When I say expert, he is an EXPERT. I hope he emails back. He says he gets around 250 emails a day. I sent him the link to here and to the EE question and also inquired into his consulting directly. He states on his site that he does consulting for a decent price and can remote in and troubleshoot. He has answered over 5,000 questions on EE just in SQL Server 2005 alone.

Not to put down AC, as I stated before, it is difficult for them to help with the networking part. They are trained in the application and are at a disadvantage in that they probably aren't proficient at networking and possibly not even the inner workings of SQL.

I could tell you a story about my first month with AC and my partner and how crazy it was. Trust me, it was worse than what you are experiencing.

I am still not sure why you don't make a backup to .enc, and other multiple backups, uninstall v5 from the server as well as from SQL. Uninstall SQL Express and try the trial version of the full SQL Server 2005 Workgroup. Then reinstall.

Anyway, we shall see if he emails me back. I hope if he does you will give it a shot.

HTH Thanks.

Hi Bert,

Thanks for your encouragement. I have been using AC from the Microsoft Access format to SQL Server. We grew from 2 to 7 doctors in the past 7 years. We have gone through the pain of merging two different offices AC data to one data set. The AC Version 4 using SQL Express has been functioning well for us. We have the same network and servers in the past one year. Problem started after we updated to Version 5.

I was hoping that AC tech support can help us to solve the problem. Talking to their tech, it seems that we are not the only group that is facing this issue. If you think they will not come up with a solution, I'll ask our IT to switch SQL Express to SQL 2005 next week. I hope the AC tech support can guide us through the process.

Philip

Hi Philip,

Well, I should be careful how I word things. I have had a couple of issues that support has been able to help me through. I don't want to say they can't. I am only saying that network issues are sometimes tougher than simply fixing AC issues, because they can sort of "get their hands around the AC program" while it is difficult for them to know exactly what the network is about.

Case in point. As you may know with some Microsoft servers and client OS, attaching the computer to the domain is a easy as opening your browser and typing http://connect. It then does the rest, attaching it to the domain, setting up DNS, etc. Well, the other day, I set up a Windows 2008 Server as a "sandbox" to play in. I had to manually attach it to the domain. I did this, and it showed that it was attached to practice.local and its full DNS name was Sandbox.practice.local. It could see all the computers on the network including the production server, but it couldn't connect. It turns out that the default login screen was Sandbox\username which allowed me to log in, but it was really logging into the workgroup account. Once, I realized my mistake and logged in under Server_name\username, everything went fine. So you can imagine having AC on there and trying to connect to the server. Just an example. Maybe a poor one.

Our group has been having exactly the same problem. When we switched from Version 4 to Version 5 we have had nothing but problems with terminals being dropped from the network periodically. We have up to 31 users. We have spent hours with technical support from AC and have gotten nowhere. We are using Microsoft 2003 Server/SQL database. We are usually able to get the system up and running by going to the main computer at the server, clicking on the "IT guy" and having it go through the series of checks. This is very annoying and frustrating as it is disrupting patient care and workload. The error screen that comes up on the desktop terminals is always the same "There is a problem preventing Amazing Charts from starting".... "There was a miscellaneous error while trying to start up. Please contact techical support"....ERROR Invalid procedure call or argument.
I am not a computer expert by any means but sense that our problem is exactly what is being described by others on this thread.
I am also not clear on what the solution is or should be.
Is AC aware of the seriousness of this problem?
Is the problem going to go away with the next version?
Any suggestions on who might be able to delve into this more?
Is the problem with not having a powerful enough server?
Thank you.
Stephen Mac Kerrow M.D.
Pueblo, Colorado

I think it Max connection problem on SQL Express.
The maximum limit to the number of SQL Server Express instances is 16 on the same machine. These instances must be uniquely named for the purpose of identifying them.

I doubt seriously that they have 16 instances on the same machine.

I do not think it has anything to do with your server.

Well, if it is not the server then what? Any other suggestions?
To me this seems like a pretty serious problem and I was hoping for more help and ideas from AC.

I agree. Again, I am a bit behind the curve due to not having installed v5 yet, which may be a good thing. I wish I did. If there were an issue, I could contact my new SQL guy. Unfortunately, I really think this is a SQL issue, and I personally, have found SQL a bit difficult, and it appears that AC support has as well.

When you installed AC, is SQL 2005 the default instance and AC a named instance? Not that I would be much help there? Also, have you checked the speed to all your computers using the Amazing Utilities guy. (I don't much like that guy).

I am glad (upset too) that finally another user group is confirming the problem that we are facing in the past few weeks. Our staff and IT spent hours and hours testing the network and server. Multiple calls to AC and multiple remote log in failed to solve the problem. We cannot see any limitations in our Server 2003 or the wired network. The problem seems to be the software problem of AC V5 and SQL Express. Can some one explain the 16 instance limitation? If we cannot solve the issue, we may have to switch to other EMR sadly.

By the way, even though we have this SQL connection error, the connection time at the virtual IT remained less than 1.2 seconds in every single laptop.

The only limitations for SQL Express are the 1GB limit, the use of only one CPU and the 4GB database. There are some other minor differences.

We are getting user instances and SQL instances mixed up. When you install SQL Express it asks you if you want to make a named instance or a default. It is best to choose default and you will have an instance with the name of SQLEXPRESS. This instance can be shared to many applications. On each application, you can allow it to install to that instance or have a named instance. There is really no need for a named instance unless you want a differently configured instance.

The user instances which may or may not be on as a default and my guess is they can't even be used with AC's default installation is used by user who are non-administrators. User instances allow you to move databases and connect with the user instances. While there is a 16 user instance limit, this is not affecting your ~ 30 computer network as you are only using the one either default instance or named instance. The SQL Server Express Engine can handle many instances and applications.

I am not sure what is causing your problem, but it does seem evident it is something within v5. On the other hand, many other users (albeit not as big), are not having that problem so maybe it can't be blamed on v5. Some things seem obvious, but they are not. You must have uninstalled v4 to install v5 both the only two SQL database versions.

The only limit that I see possibly affecting the perforance would be the 1GB. But, I doubt that is the problem, and I don't know why v4's RAM needs would be different than v5. Have you used Task Manager to monitor how much RAM it us using as you add more computer users.

Also, have you tried troubleshooting by starting with only a few computers and adding one at a time? Or any using the non-terminal server computers? How often do you get the drop? Is it often enough that troubleshooting in that way would work?

How often have you reinstalled v5? I think a network issue is just as likely as a program issue. It isn't like you used versions 4 through 10 only to have issues with 11. I would be a little wary of the XP Home versions, although I don't know why they would be overall problematic.

If you run v5 on just one self-contained machine are there problems? Or can you run v5 on the server? It is still connected to the same database.

I would urge you not to change EMRs only because of this issue. I understand it must be making life miserable, but there has to be a solution. My next comment will be in a separate post.

Check out the "Beta Versio[n] Slow" thread, as in exploring the SQL Issue discussed here, we may have come up with a possible explanation - too many messages in the Amazing Charts inbox.

Since V5, with the CCHIT mandated orders tracking features, messages are read into the computers memory more frequently. Practices that have a ton of messages may experience a significant delay in loading these messages. We hypothesize that this may allow the SQL Server connection to time-out, and thus appear as a dropped connection.

This doesn't explain all instances we've seen, but it may explain some - so we recommend deleting old messages from your inbox, and/or saving patient-related messages to the patient's chart. This could alleviate the issue. (One of our users having serious slow down and dropped issues has over 7000 messages in their inbox.)

Also, we have found in some instances that complex networking setups(such as Terminal Services), may be part of the issue with SQL Server dropped connections. Keep in mind that Amazing Charts is designed for small practices (1-5 providers) on standard off-the shelf computers and peer-to-peer common networks. The more complex and expensive IT-related set ups that are introduced, the more potential variables and complexities that have not been anticipated or tested will occur. And while we look into each and every issue that we learn about, answers and solutions will not happen overnight. Again, we recommend you stick with peer-to-peer networks on off-the-shelf computers +/- a server for practices with > 3 providers.

Jon

Switch to another EMR due to a problem with V5 upgrade? Converting data would be more work than simply going back to V4.

I would recommend that you implement your backout procedure and get V4 working again. If you can install V5 on a test environment and duplicate the error, then you could let AC team troubleshoot the problem on that test AC instance vs. your production AC instance.

Having the latest and greatest version doesn't do much for your practice if you can't get it working dependably. Like Bert, we are still on a previous, more stable version (although he upgrades his production version faster than even I do). There is a lot to be said for being a follower on the IT side and often this is best practice. Let other people take the arrows and learn from them.

Eric, I can't agree more. I have been somewhat involved in helping with this. I currently have v5 in a test envirnment, which, of course is not 30 computers. But, all in all, I don't think it is the load.

To me, with my many problems with networking in the past, it more like with 7 computers you have certain percentage of issues, and with 28, there is four times that many.

I will be upgrading this weekend, and I will roll out carefully.

I have suggested 3rd party help, but I have backed off a little until Jon's method has a try. Not sure why messages would matter, but, hey, it's the only answer so far.

Many users were familiar or could get up to speed with Access. There is a reason SQL is much better besides being more robust. It has to be a bit more technical.

So, if Jon's fix works, that's great. And, it is evidence they are working on the problem.

If anything, the only issue I have had with support or AC is they tend to work on problems (and come up with a solution) for too long in the background without addressing it right up front. I mean this to be looked at in a good light at least here as it seems they are listening to those who are having issues.

Hang in there everyone.

Thanks Eric.

I have begun to notice a new situation that causes AC 5.026 to quit unexpectedly. AC is open on my computer (not the main computer), and I open a folder located on another networked computer. AC will frequently just quit and will need to be reopened.

Someone has over 7,000 messages in his or her inbox? That is going to take forever to go through.

Thank you for everyone's suggestions.

Overall, I am just a physician, the end user of a EMR product, I am not a computer engineer. Going back to V4 which is not CCHIT certified is not an option. I introduced AC to our office 7 years ago because I like the philosophy of AC. And I believed that AC will make it happen. I hope that AC is not giving up on us.

We grow to a 7-doctor group in the past few years. No more than 4 doctors is working in the offices while 3 others are working in the hospital. If 4 doctors working in 2 offices at the same time is too much for AC, that means I have made the wrong choice or we are a wrong match. Please let me know if you cannot help.

I am wondering if this is a problem with sending orders. Sending orders is a new feature in Version 5. I never sent my orders before. I would print them and save them. We have discovered that sending them has filled a mailbox that doesn't need to be filled. Saving orders sends them to the tickler file....view...orders. This happens without sending the orders anywhere. For me, the program has been shutting down every time I send orders. I have no reason to send most orders but just did so because I had the option of doing that. I need to send my referrals to someone to do the arrangement but there is no reason to send most orders and they have just been piling up in a mailbox we really don't use.

I am one physician and in 2 months I have acquired 450 messages in my labdoctor mailbox due to sending my orders there. I just started using the lab orders section so that number has really accumulated in less than 2 months so I imagine that 7 physicians could really be filling up mailboxes with orders. We are not connected to the outside world so we still print orders and hand them to the patient. We use the tickler to assure that item ordered are returned to us. The old labdoctor mailbox has little use now, if any.

Ordering tests, sending prescriptions and switching between patients sometimes triggered the detachment from the SQL data. Closing and then reopen AC program will frequently cause the problem when the network is busy. Our AC data is 800MB. We will certainly try to delete messages as suggested by Jon.
Our IT has monitor the primary server (2003 with 4 GB RAM) the whole day and saw no spikes in the processor usage or RAM. We now dedicate this primary server to hold AC and the scanned patient data only. We use a second Server (Windows Server 2000) as the terminal server for the 8 clients to log in from the second office. All the laptops in the primary office and this second server then connect P-2-P to the primary server that holds the AC and import data. We switched some of the older laptop clients with newer ones that carry larger RAM and faster processors. We'll see what will happen next week when the offices are back to regular schedule.

The amount of data should have no bearing. Your imports have no bearing. It is connection of the databases with SQL and your clients.

Not having a domain and no WINS increases network traffic sometiems a lot. Probably not the issue.

If I were there. I would uninstall it after backing it and all the data up. I would uninstall SQL Express, then install SQL Server Workgroup Edition as suggested. Install it as the default instance and install AC to that same instance. With everything redone and having a much more robust SQL Server that can take advantage of the 3.2GBs of memory available, it may work much better. I am not sure how Jon's answer is viable (sounds weird, but we'll see), but if it does turn out to be a message issue which is taking up a lot of resources frequently, then you would have more resources.