CASE STUDY – High Performance Military Image Archiving System
A subcontractor to a US Military prime contractor engaged Lextel to develop a high performance image archiving system.
High Resolution Images were being captured by an airborne system, on up to 6 multi-terrabyte raid storage arrays. The project requirement was to transfer the image data from the raid arrays, onto IBM LTO tapes, using a specific format that enabled rapid retrieval of specific images from the tapes. Transfer of the entire array had to be completed in approximately 30 minutes, which was the time the airplane carrying the image acquisition system would be on the ground to offload the image data. Another requirement was to support the re-generation of the raid storage array content from the previously recorded tapes. Finally, a user friendly GUI was required for operator control of the archiving system.
Initial Analysis Work
Lextel began with a system analysis and generation of a functional specification, development proposal, work breakdown, and time estimate.
The system design ultimately consisted of from 1 to 5 IBM PC compatible systems running Windows, with IBM 5 slot LTO tape changers attached to each. The pc’s were connected to the Raid Arrays in parallel using Fibrechannel. In this case, the use of Windows and Microsoft Visual C++ was mandated by the prime contractor.
Lextel specified a GUI targeted to the needs of the system operators. This included capabilities for initiating disk to tape operations, scanning of the disk data and selection of ‘scenario’ data from the raid arrays to be transferred to tape, selection of specific tapes to be generated in a multi tape set, selectable status display as the system was running, diagnostic controls to allow the system to be tested and validated on site, and many other features.
The system was scalable in the sense that additional PC’s could be installed and operated in parallel in order to reduce the time needed to generate an entire tape set. It was also scalable in that the number of Raid arrays could vary. The software developed by Lextel accommodated all scaling and configuration options.
One difficulty arose in that the design requirements mandated that the image data on the Raid arrays were formatted using the FAT32 file system. The format of the tapes was such that image data from a single Raid file could be spread on multiple physical tapes. And, for scalability and performance reasons, we wanted to be able to transfer from multiple tapes to the raid array in parallel. Adding this up, it meant that multiple PC’s had to be able to write to a single raid file in parallel. Ordinarily, this would not be possible. A single PC would open a file on Raid, write data to it, and close it. A second, third, or fourth PC could not be doing that at the same time, or the file would be corrupted.
Lextel came up with a novel solution to this problem. Basically, we had the software scan the tapes, in advance of transferring the data to Raid, in order to determine the actual layout of files on the Raid. Then we pre-allocated and formatted the files on the Raid, without populating the data. With the files pre-allocated on Raid, it was then possible for multiple PC’s to write data into the pre-allocated files in parallel, without corrupting the file structure. This solution enabled the system to scale and meet performance and functionality requirements.
Development of the Archiving System
Lextel specified and proposed the system design to the client, and it was accepted. Then we went ahead and developed the software. As required by the prime contractor, we used Microsoft Visual Studio to do this work. In this case, use of the Windows platform proved to be advantageous as the Visual Studio environment is very easy to work with, and Lextel has experience with it on a number of projects. Although this was a ‘Real Time’ system, and had some performance requirements, we were able to meet them with Windows. There were not hard and fast ‘real time’ processing requirements, such as being able to guarantee handling of interrupts in 1 microsecond, or task prioritization requirements, or other demands often found in a true Real Time system.
Hence, this is an example of a Real Time System that could ‘get away’ with using an off the shelf environment, such as Windows. Doing so saved many man hours in development time, provided a low cost platform for delivering the end system, and provided a familiar environment for the system operators.
Image Archiving System Tasks Completed
- Initial Analysis
- Recommendation of development options
- Development Proposal include Schedule, Risk Factors, Steps
- Development of software
- Novel Approach to Populating Raid Files in Parallel
- GUI Development for operator use
- Debug, Validation, Hand-off
- Deliver project on projected schedule
- Provide cost effective solution
Background Information on this project
Please let us know how we can help you!