With millions of dollars of inventory often at stake, it is critical that many pharmaceutical labs maintain climate control at their distribution centers. Recently, a major pharmaceutical producer determined that it was at risk to electric power outages and interruptions that would endanger climate control and jeopardize products processed and stored at its centralized U.S. distribution center.
The finished products stored in that distribution complex require continuous temperature monitoring under stringent, FDA-regulated, temperature limits. Deviations outside of those temperature limits are subject to regulatory scrutiny as well as arduous documentation. Any power interruption in excess of 15-30 minutes could have a significant impact on the normal operations and trigger extensive regulatory and compliance issues. In this case, brief outages and utility feeder transfers were occurring with enough frequency to cause concern about the stability of the power system.
The Situation
The local utility provides electric power to the distribution substation through two 34 kV overhead lines. An outdoor distribution substation, owned and operated by the utility, steps down the voltage to 13.2 kV. The interconnection between the distribution center and the utility occurs at 13.2 kV through double-ended, main-tie-main switchgear.
“Availability of both utility sources had proven somewhat problematic due to various utility system conditions,” said the on-site power systems manager. “Five years of data showed that one of the two utility lines would go down with a mean-time-between-failure of five to six weeks.”
Loss of one of the utility lines created a substation voltage drop on the distribution center power system. This triggered an extensive operational response from electrical personnel in order to minimize the risk of exposure to the finished product. However, if both utility lines became unavailable, the operational response would also require support from the personnel responsible for product management and compliance, as well as other building systems (HVAC, refrigeration, facilities monitoring, etc.). The manager of the distribution center’s power system operates it as a utility, billing various departments for usage. “There are thousands of people working at the combined facilities in the area,” the manager said. “If they have a problem with the power system, they don’t call the utility they call me.”
Besides utility disruptions, the manager was concerned about the internal distribution system. An extensive outage had already occurred due to insulation failure. The resulting ground fault current traveled through four circuit breakers and was detected by the trip unit on the 480 V feeder breaker in the substation, which tripped. It took more than four hours to locate the source of the fault and reestablish power to affected branch loads that included critical refrigeration units loaded with valuable finished products. Fortunately, the electrical problems were resolved during the night of the outage, and the direct costs incurred for electrical equipment were minimal. The costs incurred to address all of the regulatory implications and requirements, however, were estimated at over $100,000.
The Solution
As a result of this and other incidents, the pharmaceutical house adopted a strategy for reducing exposure to business risk. That strategy was to upgrade and automate the utility interconnection substation. That project entailed improving system protection and coordination, retrofitting the major power distribution panels and cables at the 480 V level, and the installation of a 2 MW emergency generator to serve the critical refrigeration units in case standby power was needed. The total value of these investments was approximately $2 million.
As part of the upgrade, the power systems team decided to replace the electromechanical protective relays at the main substation. While those relays had performed adequately in the past, there were concerns about the capabilities of the electromechanical relays to properly recognize reverse power and reverse current situations in instances when the utility claimed that the relays should not have tripped the incoming line (main) breakers. It was, therefore, necessary to more thoroughly monitor the tripping sequence to determine the root cause of events occurring on both the utility side and the distribution center side of the system. After evaluating products from a number of microprocessor-based relay vendors, it was decided to implement a solution using relays from Schweitzer Engineering Laboratories Inc.
To implement optimum protection, control, and data retrieval, SEL 351S relays were installed on each of the feeder breakers, bus-tie breaker, emergency generator breaker and two bus main switchgear breakers. An SEL 7000 Integrated Substation System was installed in a separate communications panel. The panel contained the communications equipment and the human-machine interface (HMI) system. SEL engineering consultants provided relay logic programming, the HMI, commissioning support, and training.
“The SEL relays were connected to a local human-machine interface computer through an SEL 2032 communications processor,” said Saurabh Shah, business development manager at SEL Systems and Services Division (SSD) in Charlotte, North Carolina. “The HMI computer provides an operational tool for remote switchgear breaker operation, gathering system data, recording system disturbances, and managing system operation. The special control schemes incorporated into the SEL relays include the service restoration scheme for the two utility feeds, fast bus tripping scheme to provide bus differential protection without separate relays, and an emergency generator starting and loading scheme.”
“One of the things the power system manager wanted to do was reduce the equipment on the door of the switchgear,” said Jim Glasser, an SSD protection engineer who handled the relay settings. “With the SEL 351S relay, we were able to replace everything on the switchgear door with just the relay and test switches. The relay features pushbuttons on the front that can be used to program open and close functions for the breaker, as well as other operations.” Two auxiliary pushbuttons operate independently of the relay to provide trip and close in the event that the relay lost power or malfunctioned.
The SEL equipment was provided by high-voltage specialists, A-Star Electric of Elk Grove Village, Illinois, an SEL distributor that was involved in reviewing system requirements, design, and specifications.
Connecting the emergency generator to equipment that was determined critical for normal operations represented a major challenge. Compressors, evaporators, controls, and monitoring equipment serving the essential coolers, freezers, and manufacturing equipment were spread over a very large area.
“Rewiring all the equipment into one emergency panel, backed up by the generator was an expensive proposition,” Shah said. “It was determined that a more efficient solution would be to procure a larger sized generator, capable of backing up the entire center and to connect it to the same substation providing power from the utility sources. So, the distribution center installed a 2 MW emergency generator, which is protected and connected to the system by an SEL 300G Generator Relay.”
Also added to the automation solution was an SEL-3010 Event Messenger, a device used to receive station alarms and provide event notification by telephone. “Whenever important events occur, the HMI will detect the event and then dial a phone number and report that event to somebody who needs to know about it,” Shah explained. “There is a sequence of numbers that can be dialed to report the event, to make certain that somebody gets the message. In this instance, it calls the power system manager and then the other engineers. This could be a telephone voice message or a pager message.”
Access to detailed information has been a very important benefit of the automation project for the distribution center’s power system upgrade team. In particular, the SEL relay’s Sequence of Events Recorder (SER) has been an invaluable source of exact data, even when minor events occur. Because the utility is not required to report “momentary” outages (those under one minute), a feeder line switch would often go unnoticed by the utility. The SER, however, would notice even the briefest event, that would cause a trip and reverse power condition.
“That is very important because the power systems manager wants to know what happened at the exact second,” said Shah. “Now, because of the SER, he can tell the utility its feeder went down at the precise moment and would like an explanation. If they ask how he knows, he can tell them, ‘Because I am monitoring your line through my relay.’ So, the SER has become an important tool.”
Future Benefits
The capabilities of the new protective relays are likely to provide the distribution center with important future benefits as well. With the existing electric utility contract expiring soon, the need has arisen to fully understand the load profile for demand and energy, as well as the actual power factor. The SEL relays were programmed to provide demand, energy, and power factor data. This information can play an important role in negotiating the new utility contract and in future capital investments for power factor correction.
“The distribution center is trying to save energy and reduce utility costs by making better decisions on energy usage,” Shah said. “They have already asked SEL, ‘If I gather all of this data, can I get the real-time (energy demand) curve, so that I can make it available to our operators who operate a lot of heavy equipment like boilers and chillers, which are really power-hungry?’ We are very hopeful that when it’s 95 degrees outside and the distribution center needs to throw in another chiller, they can delay starting the chiller for another 30 minutes or use a steam chiller or precool some of the buildings at night so that they can avoid setting a new peak demand level or incur a power factor penalty with the utility. If it’s the middle of the month and they know that electric usage is already running high, they will be in the position to do some creative things that will help avoid the penalty curvewhich would be a major savings opportunity.”
