This is a case study of the need to understand how flash drives function, and for choosing the right drive for the application. This study discusses how we blew up multiple drives, how we hunkered down to understand the reason, how we resolved the issue, and how we arrived at a guide for choosing flash drives.
Srinivasan holds a Bachelor’s Degree in Electronics and Communication Engineering from Anna University-Chennai, a Master’s Diploma in Business Administration [MDBA] from Cambridge University, and has received his Indian Management Fellowship [IMF] from Cambridge University as well. An Embedded product design strategist and technology evangelist at heart, Srini heads the Industrial Automation Center of Excellence at Sosaley Technologies.
You might be thinking – What? A pen drive bursting? Are you kidding me? Is this even possible?
Here is photographic proof. We did punch a good sized hole in more than one pen drive!
The Need for A DataLogger
Armstrong, for whom we developed the Datalogger, had difficulties in collecting information on the MODBUS network. They decided they wanted a Datalogger developed that will suit their requirements exactly. In about six months’ time, we developed a product that met the specifications they were looking for. For more details on the product, click here.
The Sosaley Datalogger is a device we developed and is used in hostile industrial environments. It works on a MODBUS network that uses high voltage most of the time. The industrial environment in which the Datalogger is used has large machinery that consumes very high current. In addition to power spikes and EMC, the product is also prone to physical abuse including vibration, hot air, and some fumes from engines.
Hey, Your Pen Drives Are Exploding!!
Considering the complexity of the usage environment, we did provide a USB protection on the circuit itself to handle such issues. We were confident this would do the trick. Little did we know how wrong we were. Within 24 hours of usage, we were looking at a pen drive that had a hole in it and complete loss of data. The first time we heard about it, we were quite shocked. This was a first time for us.
We then started investigating the issue and trying to understand why this was happening.
The Investigation Begins
When the Datalogger was delivered and installed, we received complaints that some of the pen drives were working, while some were going bust. We decide to investigate the matter. We took two pen drives from different brands that had burst and opened them up.
After careful scrutiny, we narrowed the issue to both the pen drives having the same micro-controller inside. We now had some data points to work upon. We raised an issue with the manufacturer. After a series of mail exchanges, the manufacturer agreed that the particular microcontroller used in both drives was not capable of working at temperatures beyond 50° Celsius.
The assumption that was made during the design of the drive and the micro-controller was that such pen drives will be used on devices mostly in an air-conditioned environment. The other assumption was that these pen drives will not be attached or used for a long time. They will be used in a device for a short time, ejected, then either used on another device or kept aside. Most important was that these devices were fed electric current that has very little spikes, and voltages that were managed.
Against this, we were subjecting the pen drive to long unattended usage, and to an environment where the average temperature was 80 degrees! In addition, the environment had high current and sudden surges and spikes every time a compressor or a motor started.
Understanding the Usage Environment
We understood a number of things at this stage. The most important was to understand the environmental variables that are present before deciding on a piece of hardware.
1) Flash Drive And Data Type Are Interlinked
If you need to store and transfer video files, it is important to look at high transfer speeds. If you need to create a boot disk, the size could be small, but endurance has to be high. If you need to store and transfer confidential data, it is imperative you look for drives with encryption built-in. If the files are ordinary office documents and not critical, speed is unimportant and size may be considered.
In our case, the Datalogger reads and writes on the drive on a continuous basis. Though the size of each record is small, the drive is in constant use and every byte is important. Reliability, the safety of data, operating temperature and tough industrial quality is very much needed.
2) Read & Write Cycle
It is critical to understand the read-write cycle – what it means, and how it is relevant to you.
All flash drives have a memory where the data is stored. These drives are called solid drives. In solid drives, there are no moving parts but contain blocks of non-volatile memory. Each of these blocks contains a number of individual cells that hold the bits of data. These bits are turned off or on through an electric charge. How these cells are organized becomes important depending upon the usage of the flash drives.
|Triple level cell, or TLC, stores 3 bits per cell and is the least expensive form of flash to manufacture. This is the one that is used most commonly in consumer drives. TLC has the slowest read-write speeds and the lowest read-write cycle of 3,000-5,000.||At the next level are multi-level cells or MLCs. These store 2 bits per cell and have slightly faster read-write speeds. Their read-write cycles are 8,000-10,000. There is an enterprise version of MLCs called eMLC that also store two bits per cell. They have faster read-write speeds and a read-write cycle of 20,000-30,000.||Finally, we have the single level cells, or, SLC. They store one bit per cell, have the fastest read-write speeds and a read-write cycle of 90,000-100,000.|
Flash drives made with SLCs are quite expensive, sometimes running to 10x of those made with TLCs. In many areas of application (such as our Datalogger), they are the only choice.
3) The Need For Speed
As explained above the speed of the drive is related to the cells used. Your standard consumer drive uses TLC and is the slowest. In most cases, they work with USB 2.0 and have a theoretical max speed of 480 Mbps. If you want to transfer a 16GB file, this will take 9 minutes or more. The same flash drive using USB 3.0 can do it in less than a minute.
The actual throughput performance of the USB device varies based on your computer’s settings. In a typical well-conducted review of a number of high-performance USB 3.0 drives, a drive that is supposed to read files at 68 MB/s and write at 46 MB/s, could only manage 14 MB/s with large files and 0.3 MB/s with small files. When combining streaming reads and writes, the speed of another drive that was specified reads at 92 MB/s and writes at 70 MB/s, was just 8 MB/s. These speeds differ radically from one drive to another. Some drives could write small files at just 10% of their speed for large ones.
If your need is critical and the data is one-time only, it is important you understand these in detail and choose the correct drive.
4) File System
What is a file system? A file system is a way an OS stores data in its memory and drives. A file system, in general, consists of data split into blocks and stored in multiple locations on the drive. The OS indexes these locations, forms a directory structure, and reads the data sequentially or randomly depending upon the file type and user needs.
OSs such as Windows, Linux, MacOS can work with multiple file systems to maintain compatibility. At the same time, they have an internal file system that makes it necessary to convert data from one file system to another. When you format a drive, it is formatted to understand the native file system, unless you choose another file system.
It is hard to argue or convince one on the pluses and minuses of the file systems. In general, if you are using a particular OS, it is safe to use its native file system. You can always convert data from one file system to another without losing even a single bit.
Most common and popular files systems are FAT32 (data transfer not more than 4GB), exFAT, NTFS, APS, etc.
5) Usage Environment
In a hostile environment, it would be foolish to use a regular commercial flash drive. In these environments, what is needed are the following:
- The drive and USB connection must have a very strong grip. An extra clip would be very nice.
- The drive and the device using the drive must be capable of withstanding constant vibration.
- The drive should, preferably, have a metal casing and some form of non-conductive protection.
- The pen drive should have been manufactured with RoHS compliant systems.
- The pen drive must be capable of high read-write cycles and guaranteed not to lose data. Data retention is critical.
Special drives are available from pen drive manufacturers that meet these requirements.
6) Operating Temperature
Operating temperatures play a major role in selecting the pen-drive that matches the application needs. Our client had used commercial grade pen-drives that were not intended to work at more than 50° C
We made some changes to the hardware to give the USB drive additional surge protection. We built a small external circuitry to measure the actual temperature of our client environment where the Datalogger was installed. We ran the Datalogger 24/7 for a few days and measured the temperature range. Our temperature logs showed that the temperature went up to 85°.
Using this as the baseline, we procured pen drives that could stand that temperature and more. With that installed, our Datalogger worked perfectly.
The research-oriented minds of our engineers, our investigative curiosity, and some amount of stubbornness helped us in understanding the issue and delivering a good resolution. The client is now confident that the Sosaley Datalogger will work for long periods in a harsh environment and not lose any data.
We will use these guidelines in the future, not only for pen drives, but for all hardware and software.