Seminar: Data Archiving

Getting Started (details)

The key with starting this process is to start the process. Do not wait until tomorrow. Do not allow the procrastination monster to rear its ugly head. Get a simple backup system started and start by adding archiving to your normal activities. When you go online to pay a bill download the statement and save it in your archive. If you get digital receipts through your email save them in your archive.

  • Do NOT just talk about archiving — get started today!
  • Start by organizing and saving digital data as you do your normal routines
  • Consolidate existing data onto a single archival system with backup
  • Digitize and organize data starting with the Worst First

Select Backup System & Type

You need multiple backups to adequately protect your data from disaster. Your computer normally stores your working data such as photos, documents, and data files. You must first protect your data in case your computer fails. There are a variety of backup systems including:

  • External Media such as CDs or DVDs are cheap and reliable but have limited storage capacity.
  • Thumb Drives have limited storage but can work well for your critical data such as annual data.
  • External Drives have much more storage capacity and are small and relatively inexpensive.
  • Cloud and Internet offer large external storage but are only as good as the service provider.

Multiple Backups are required regardless of the backup method used. I prefer physical backups rather than using the cloud. If you use an external drive then you MUST have multiple back-ups. Personally, I have my working data in my computer, a local backup drive, and another backup drive in a safe-box at the bank. You should always keep one backup on-site and one off-site.

Next select the type of backup for your situation.

  • An image backup backups-up everything on a hard drive including system files, boot sectors, system files, hidden files, applications, data files, etc. An image backup is useful for restoring a hard drive after a catastrophic failure. You must generally have exactly the same hardware to restore a hard drive image.
  • In a full backup all files are copied to the backup device. The files can either be copied as is or compressed. Copying the files uncompressed takes the most time and space but the files can be accessed directly.
  • An incremental backup starts with a full backup. On the next backup the files in the previous backup are compared with the current files to backup. The backup contains the files that were added or changed since the last backup.
  • A rotating backup uses multiple full backups stored by date. Keep 3-5 multiple full backups. The data sets are backed-up on the same drive and the entire drive is backed-up and stored off-site.

I use multiple USB hard drives and create rotating full backups. One backup is kept on-site and one off-site. Storage is small and inexpensive so I prefer the ease and versatility of having multiple full backups. I keep three sets of dated backups per drive.

Folder Structure

Next, create a folder structure for your files. Creating a good folder structure from the start makes it faster and easier to organize your data. Image if you were going to organize a mountain of papers into a file cabinet. You could start by labeling file folders and organizing in the file cabinet drawers. As you file papers you would then place the papers in the appropriate file folder. With your digital data you can create folder structures and copy your digital files into the proper folder. If you would like a starting place review my suggestion for Digital Templates. Customize these templates for your needs. I have folder templates for annual data, personal data, and assets.

I created folders with each year and within each year I have folders for healthcare, taxes, education, investments, receipts, etc. After I pay a bill I download a statement and store it in the archives. This way I have copies of statements and I spent very little time getting the digital statement since I was on the website to pay the bill anyway. You must be very careful in how you name the files but more on that in a bit.

Scanning (details)

Many of your documents need to be scanned to get them into digital files. Before scanning create a folder structure to organize your digital files. Scan each document and move it to the arhive. Mark documents as “scanned” and verify that each document was digitized properly. This verification step may seem tedious but scanning problems can occur, such as: (1) document was not readable, (2) original document misfed during the scanner, (3) document was skipped because it stuck to a previous page, etc. Do NOT discard ANY scanned document unless it resides on both the primary and the archive data drive.

You probably have lots of documents that you would like to preserve including:

  • Tax records, forms, receipts, home/business records
  • Photos and videos
  • Birth certificates, passports, social security cards
  • Home documents including deed, mortgage papers, plat
  • Receipts for appliances and general purchases
  • Certificates and other records
  • Drawings, school work, and other memorabilia

File Naming

A major goal of organization is making your information easier to find and access. Part of making the information easier to access is naming the scanned files in a logical manner. When you view the filename you want the related files to be grouped. Dated material should be store by date and category. My personal filename rules include:

  • NO spaces in the filename
  • Use a period ONLY before the file extension
  • Separate name components using an underscore
  • Use camel case notation (for example: HomeDepot)
  • ALWAYS use two digits for month and day
  • ALWYAS use four digit year

Filenames should be in a consistent naming format. When viewing a sorted list of files the related files should be grouped. When the files are listed alphabetically all of the same bank records, credit card statements, etc. should be grouped. You may also be able to store the statements for an entire year in a single file. If you are scanning printed documents you can place multiple scanned pages in a single digital file.

For bank and credit card statements I use this format: accountName_yyyy.{extension}. The statements for this bank or credit card will appear together when sorted. Example:

amex_2016-01.pdf → for the January 2016 statement for American Express

Normally receipts are stored by date. Each item on a receipt should also be entered into your accounting software so that you can cross-reference the receipt with the accounting notes and date. I use this format: yyyy-mm-dd_source.{extension}. Examples:

2019-02-03_Walmart.pdf
2016-03-04_HomeDepot.pdf

Tax forms are normally stored by date and form type. The dates can be year only, year with quarter, or complete date. Examples:

2017_Q1-ExciseTax.pdf
2016-01-26_IrsLetter.pdf
2015_1040-ScheduleC.pdf

Copies of checks can be stored by year and check number. You may want to limit the number of checks per file when scanning checks individually in this format: yyyy_checks_{start#}-{end#}.{extension}. Example:

2012_checks_1234-1350.pdf

ALL dated files should include the year so that filenames will be unique. If you copied files from different years into a single directory (for an insurance claim, for example) then you would not have duplicate filenames.

Redundancy (details)

Redundancy is the key to securing data using backups and archives. When designing a backup system put on your pessimist hat and think of all the things that could go wrong with your data. Then design a system to minimize risks at every failure point. One CRITICAL requirement of an archival system is multiple levels of redundancy. The first line of defense is protecting against catastrophic data loss. These types of losses can be caused by fire, theft, or hardware failure. ALWAYS have your data in at least two places with one stored offsite.

You should have your data well protected in case you accidentally delete a file or folder. I HIGHLY recommend using rotating backup folders that use the date in the folder name (for example _backup_2018-02-01). You should maintain 2-5 backup sets. When you have too many backup sets then backup your current data and erase the oldest backup directory.

Accounting Software (details)

Your accounting software should integrate with your archival system. Receipts are normally stored by date and accounting entries include the date. When entering an accounting entry use the Comment field to describe the item and the category to classify the item. You want the items to be easy to find. When adding an accounting entry split the transaction and add a category and comment to the line items. After a house burglary I searched and submitted copies of the receipts.

Other Assets (details)

When archiving your digital assets do not forget about application data, configuration files, and databases.

Many applications store data on your computer and you have to be sure to backup that data. Applications such as Quicken, iTunes, etc. allow you to choose the directory that they use to store your data. Set these applications to store their data in a directory that is part of your backup procedure. Some applications store configuration details and you may also want to save a copy of the configuration to your archive. You can often export the configuration to a file and save it in your archive.

If you do development work then be sure to export your databases and store them in your archival system. I have a directory for website development that includes a _backup directory for each project. This backup directory contains database and other backups for that project. When I copy the entire development directory to the archives it includes the website files and the backup copies of the database data.

It is good practice to have your websites backed-up. The server companies may maintain backups but I like having my own backup copy of my websites. Most of my websites use WordPress and there are a number of excellent plugins that make backups very easy. I use All-in-One WP Migration. This plugin is normally used when moving a WordPress site but it creates excellent backups. This plugin saves all themes, plugins, configuration, and database tables for your entire website. It combines and exports all of this data as a single file. If you need to rebuild your entire website you just install a clean copy of WordPress, add the plugin, and then use the plugin to import your backup file. If you change your domain name this plugin also updates the database entries so that they work with the new domain.

Summary

  • Keep your data in a very limited number of folders on your computer. Have your applications store their data in one of these folders so that you automatically copy program data when doing a backup.
  • Create a consistent directory structure for your annual data, personal data, and asset details. Do NOT keep this data on the hard drive of your working computer. If your computer is stolen you do NOT want this data in the hands of a criminal.
  • Use a logical and consistent format for filenames. The “like” files should be grouped when you view a sorted list of files.
  • Keep a set of rotating backups on two external drives. Always store one backup drive off-site.
  • Save copies of statements as you pay bills. This way you are expanding your records while expending very little time and effort.
  • Use accounting software to help keep purchase records. Use categories and comment fields to label purchases. Store receipts by date that reference back to your accounting records.
  • Be sure to backup other digital assets including databases, configuration data, etc.
  • Create your checklist that describes your personal backup routine.