Paul Hayes's List: Data Wrangling

• Computer storage is measured in bytes, kilobytes (KB), megabytes (MB), gigabytes (GB) and increasingly terabytes (TB). One byte is one character of information, and is comprised of eight bits (or eight digital 1's or 0's).
• For example, an hour of DV format video footage consumes about 12GB of storage. Non-compressed video requires even more space -- for example 2GB for every minute of standard definition footage, and 9.38GB for each minute of non-compressed 1920x1080 high definition video.
• Except where two internal hard disks are considered essential on the basis of performance (and possibly convenience), a second hard disk is today most advisably connected as an external unit, or what is sometimes now known as a "DAS" or direct attached storage drive. DAS external hard disks connect via a USB, firewire or an E-SATA interface (see the hardware section), with USB being the most common.
• Almost all optical storage involves the use of a 5" disk from which data is read by a laser. Optical media can be read only (such as commercial software, music or movie disks), write-one, or rewritable, and currently exists in one of three basic formats. These are compact disk (CD), digital versatile disk (DVD)and Blu-Ray disk (BD). A fourth format called High-Definition DVD (HD DVD) is now dead-in-the-water.
• Solid state storage devices store computer data on non-volatile "flash" memory chips rather than by changing the surface properties of a magnetic or optical spinning disk. With no moving parts solid state drives (SSDs) -- are also very much the future for almost all forms of computer storage.

3 more annotations...

• if you download it in standard quality, almost always it will be 700mb. People like that size because it fits on a CD. HD tends to be around 1.8 to 4.7 Gb. 1080p HD can go up to 28GB (bluray quality)
• Other
• Depends on the file extension .. sometimes a movie would take 700 MB

  or 250 MB for the same movie!
  

   It's not a matter of the movie length only, file type matters too.
• matters
• Most movies are around 650-750 mb which equals like 3/4 of a gb
• 1.5 to 3 Gigabytes.
• It depends on the quality, but expect it to be similar to a DVD, or about 4.7 GB. max.

5 more annotations...

• Bit 

  A bit is the smallest increment of storage on a computer. Imagine each bit is like a light bulb. Each one is either on or off, so it can have one of two values (either 0 or 1).

   Byte 

  A byte is a string of 8 bits (eight light bulbs in a row). A byte is basically the smallest unit of data that can be processed on your family computer. As such, storage measurements are always done in bytes rather than bits. The largest decimal value that can be represented by a byte is 2⁸ (2 x 2 x 2 x 2 x 2 x 2 x2 x2) or 256. For more information on binary numbers, including how to convert them to decimal, please see the resource area below.

   Kilobyte (KB) 

  A kilobyte in binary is 1024 bytes(2¹⁰), but it also used to refer to 1000 bytes (the decimal interpretation). This is where things start to get really confusing! You can see that a binary KB is slightly bigger than a decimal KB.

   Megabyte (MB) 

  A megabyte in binary is 1,048,576 (2²⁰) bytes. In decimal it’s 1,000,000 bytes (10⁶).

   Gigabyte (GB) 

  A gigabyte is either 2³⁰ (1073741824) bytes or 10⁹ (1 billion) bytes. By now the difference between the binary version and the decimal version is quite significant.

• That depends. Most songs are usually under 10 MB so you could probably put up to 20 songs on

  there.
• Avg song size is 4-7MB.
• Most songs are usually somewhere between 2 and 5MB, depending on the compression, bit rate, length etc...

1 more annotation...

• The rule that many people go by is 1GB=500 songs, 2GB=1000 songs, 4GB=2000 songs...

• The range of memory and storage within and attached to a computer system is known as the Storage Hierarchy and to help understand this further can be categorised into 4 segments.
• • Primary Storage is the top level and is made up of CPU registers, CPU cache and memory which are the only components that are directly accessible to the systems CPU. The CPU can continuously read data stored in these areas and execute all instructions as required quickly in a uniform manner. Secondary Storage differs from primary storage in that it is not directly accessible by the CPU. A system uses input/output (I/O) channels to connect to the secondary storage which control the data flow through a system when required and on request
   
  • Secondary storage is non-volatile so does not lose data when it is powered down so consequently modern computer systems tend to have a more secondary storage than primary storage. All secondary storage today consist of hard disk drives (HDD), usually set up in a RAID configuration, however older installations also included removable media such us magneto optical or MO
   
  • Tertiary Storage is mainly used as backup and archival of data and although based on the slowest devices can be classed as the most important in terms of data protection against a variety of disasters that can affect an IT infrastructure. Most devices in this segment are automated via robotics and software to reduce management costs and risk of human error and consist primarily of disk & tape based back up devices
   
  • Offline Storage is the final category and is where removable types of storage media sit such as tape cartridges and optical disc such as CD and DVD. Offline storage is can be used to transfer data between systems but also allow for data to be secured offsite to ensure companies always have a copy of valuable data in the event of a disaster.

• Megabyte = 1/1152921504606846976 yottabyte
    = 1/1125899906842624 zettabyte
    = 1/1099511627776 exabyte
    = 1/1073741824 petabyte
    = 1/1048576 terabyte
    = 1/1024 gigabyte
    = 1 megabyte
    = 8 Megabits
    = 1024 kilobytes
    = 8192 Kilobits
    = 1048576 bytes
    = 2097152 nibbles
    = 8388608 bits
• gigabyte = 1/1125899906842624 yottabyte
    = 1/1099511627776 zettabyte
    = 1/1073741824 exabyte
    = 1/1048576 petabyte
    = 1/1024 terabyte
    = 1 gigabyte
    = 1024 megabytes
    = 8192 Megabits
    = 1048576 kilobytes
    = 8388608 Kilobits
    = 1073741824 bytes
    = 2147483648 nibbles
    = 8589934592 bits
  
• terabyte = 1/1099511627776 yottabyte
    = 1/1073741824 zettabyte
    = 1/1048576 exabyte
    = 1/1024 petabyte
    = 1 terabyte
    = 1024 gigabytes
    = 1048576 megabytes
    = 8388608 Megabits
    = 1073741824 kilobytes
    = 8589934592 Kilobits
    = 1099511627776 bytes
    = 2199023255552 nibbles
    = 8796093022208 bits
• petabyte = 1/1073741824 yottabyte
    = 1/1048576 zettabyte
    = 1/1024 exabyte
    = 1 petabyte
    = 1024 terabytes
    = 1048576 gigabytes
    = 1073741824 megabytes
    = 8589934592 Megabits
    = 1099511627776 kilobytes
    = 8796093022208 Kilobits
    = 1125899906842624 bytes
    = 2251799813685248 nibbles
    = 9007199254740992 bits
• Exabyte = 1/1048576 yottabyte
    = 1/1024 zettabyte
    = 1 exabyte
    = 1024 petabytes
    = 1048576 terabytes
    = 1073741824 gigabytes
    = 1099511627776 megabytes
  

    = 8796093022208 Megabits
  

    = 1125899906842624 kilobytes
    = 9007199254740992 Kilobits
    = 1152921504606846976 bytes
    = 2305843009213693952 nibbles
    = 9223372036854775808 bits
• Megabits
• 1125899906842624
• Zettabyte = 1/1024 yottabyte
    = 1 zettabyte
    = 1024 exabytes
    = 1048576 petabytes
    = 1073741824 terabytes
    = 1099511627776 gigabytes
    = 1125899906842624 megabytes
    = 9007199254740992 Megabits
    = 1152921504606846976 kilobytes
    = 9223372036854775808 Kilobits
    = 1180591620717411303424 bytes
    = 2361183241434822606848 nibbles
    = 9444732965739290427392 bits
• yottabyte = 1 yottabyte
    = 1024 zettabytes
    = 1048576 exabytes
    = 1073741824 petabytes
    = 1099511627776 terabytes
    = 1125899906842624 gigabytes
    = 1152921504606846976 megabytes
    = 9223372036854775808 Megabits
    = 1180591620717411303424 kilobytes
    = 9444732965739290427392 Kilobits
    = 1208925819614629174706176 bytes
    = 2417851639229258349412352 nibbles
    = 9671406556917033397649408 bits

7 more annotations...

• An exabyte of data is created on the Internet each day, which equates to 250 million DVDs worth of information.
• And the idea of even larger amounts of data — a zettabyte — isn’t too far off when it comes to the amount of info traversing the web in any one year.
• Cisco estimates we’ll see a 1.3 zettabytes of traffic annually over the internet in 2016 — and soon enough, we might to start talking about even bigger volumes.
• After a zettabyte comes yottabytes, which big data scientists use to talk about how much government data the NSA or FBI have on people altogther
• Put it in terms of DVDs, a yottabyte would require 250 trillion of them.
• Connected Intelligent Solutions at Intel, we now know the next-generation prefixes for going beyond the yottabyte: a brontobyte and a gegobyte.
• On YouTube, 72 hours of video are uploaded per minute, translating to a terabyte every four minutes.
• 500 terabytes of new data per day are ingested in Facebook databases.
• Sensors from a Boeing jet engine create 20 terabytes of data every hour.

7 more annotations...