A letter from Sean McGoldrick
Now that the immediate crisis associated with last week's intermittent power outages are over, we would like to provide a more comprehensive explanation of what has been occurring with our campus electrical system.
Since February of this year, we experienced multiple power outages on our 25-year old "south-loop" electrical line that typically lasted about three to four hours. Initially we felt the first two outages which occurred in February and April were the result of voltage issues caused not by our system but by our electrical provider NV Energy. We subsequently met with NV Energy engineers and senior leadership to review the outage data and, ultimately, both parties agreed to continue to work together to identify the cause of the problem.
There was a relative lull during the summer where we had reliable power but then again this fall, we had two more outages on the same south loop which manifested itself in the same fashion as the spring outages; fuses would blow, we would test our lines and they appeared okay, no singular piece of equipment would burn up and thereby identify itself as the culprit in the outage, and we would finally install new fuses, energize the line and power would be restored. Our ongoing concern was that while we were able to restore power, we could not identify the root cause of the problem. After the third outage this fall, we started to believe our system was the cause of the power outages. The proximity of the fourth outage was only days away from the third outage, and was a confirming event that our system was the problem, and we developed a theory that we had a very tiny crack in either our cables or connections that allowed electricity to go to ground and shut down the system.
During the outage that started on Sunday, Dec. 2 at about 9:30 AM, we began a systematic process to isolate various buildings in order to test and locate the fault. We worked until early evening but similar to previous outages, we could not locate a fault.
On Monday morning, we experienced another outage. We moved four facilities, the Joe Crowley Student Union, the ARF, the Judicial College and Knowledge Center off the problem south loop onto the north loop in order to isolate the potential problem area to a fewer number of buildings. We also hired a local firm with more sophisticated equipment. The firm tested various electrical lines and connections, and while we were able to restore commercial power after about five hours, we still had not discovered the root cause of the outage by the close of business on Monday evening.
Tuesday morning started with another outage and once again, we took the opportunity to expand the testing to more lines which ultimately led to a longer outage. At about 1 PM we migrated the Raggio Education Building, Brian Whalen Parking Structure and the Fitzgerald Student Services off the south loop onto the north loop to again reduce the number of potential areas we would need to test and further aide us in diagnosing the cause of the problem. Later that afternoon, our testing isolated a problem cable near the Harry Reid Engineering Lab (HREL) so we took it, along with Scrugham Engineering off the south loop and provided generator power to both buildings. We subsequently reenergized the south loop and provided commercial power to the only two remaining buildings that were still on the south loop which was Davidson Math and Science Center (DM&S) and Laxalt Mineral Research (LMR). Our vendor began the task of locating replacement cable for the HREL repair and they had early indication that replacement cable was located in Los Angeles and it might be delivered by Thursday. By about 4 PM on Thursday, all buildings had power restored via generator or commercial power and we thought we had identified the problem for the previous outages.
Wednesday morning brought unwelcome news at we had the third power outage of the week on the same "south loop" servicing only two buildings: DM&S and LMR. This outage confirmed to us that our previous diagnosis on Tuesday, that the Harry Reid Lab cable was the root cause of the problem, was probably incorrect. We replaced the blown fuses at our main switch gear and reenergized the south loop at about 11:30 AM to bring DM&S back on commercial power; we left LMR off commercial power in order to trouble-shoot a suspected problem transformer. This testing proved fruitless so we continued testing elsewhere and late in the afternoon, we discovered faulty switch gear behind DM&S which we then believed was the cause of all the previous outages. We supplied generator power to both DM&S and LMR at around 5 PM and began to locate parts for the next day's repair.
On Thursday we were able to secure replacement parts and completed interim repairs to the DM&S switch gear that will hold us over until it is replaced during the Christmas break. When the work was completed, we restored LMR onto commercial power and we left DM&S, SEM and Harry Reid on generator power. The replacement cable for HREL also arrived on Thursday so we continued the work and testing needed to install the cable.
On Friday, our staff reported to work at 4 AM to begin the two hour process to get DM&S off generator and onto commercial power. We unfortunately experienced a safety incident and delayed the work until early Friday evening so as to minimize any impact on classes being held in DM&S during the day. Our contractor continued his work on running new cable at HREL and they were ready to energize the line by late Friday afternoon. We subsequently coordinated a plan with campus leadership to return commercial power in stages to DM&S by about 6 PM; the Harry Reid Lab by 6:30 PM and SEM by 9 PM. This process went very smoothly and we left late Friday evening believing we had resolved the power outage issue that had plagued us since February.
On Sunday at 1:30 PM, the south loop went down and testing began again. We quickly discovered the switch gear that was installed during the DM&S construction four years ago had also failed. Fortunately, we had a replacement switch gear on hand for work that was scheduled elsewhere on campus so we replaced the faulty gear by 11:30 PM on Sunday evening. At this point we are confident the south loop problem has been rectified.
Finally on Tuesday of this week we had a short, planned outage of SEM and HREL at 5:40 AM in order to place them onto the north loop since critical testing was occurring later this week and it was felt the north loop offered more reliable power until the south loop is completely repaired over the Christmas break.
Stable and reliable electrical power is absolutely critical in a research facility like the University of Nevada, Reno. We, like many universities across the nation, have an aging infrastructure requiring substantial funding for repairs from what is typically an exceptionally competitive but limited funding pool. The University has been very aggressive about pursuing funding to repair our system starting in 2006 with a $17M request to the State Public Works Board and more recently in our 2013 Capital Improvement Program (CIP) request of $8.8M to upgrade our 4kV to a new 25kV line. Unfortunately because of other higher system priorities these projects did not receive funding. Notwithstanding the difficulty in securing CIP funding, we have been able to secure alternate state and University funding and have available $1.2M for electrical system upgrades over the next 12 months. Additionally, we have a phased electrical system upgrade plan that is programmed in excess of $500K in 2013 from additional state and internal University funding. In sum, we believe we are positioned with the strategy, personnel and funding approach over the next several years that will ensure a stable and reliable electrical power supply.
In closing, it's been a difficult past week for those faculty, staff and students who endured the inconvenience of multiple power outages over many days. Our staff was exceptionally responsive along with various contractors in trying to locate a fault over a very complex electrical system. We learned that we need to do a better job of communicating with the campus and we've already begun that process. Finally, we wanted to extend our thanks for the patience and support that many showed last week as it proved very helpful during what was a tough week for everyone involved.
Associate Vice President Facilities Services