Document Retention Policies

CurlyNerd0813

Member
Joined
Aug 1, 2018
Location
Columbus, OH
Not sure if this really belongs here or in the General Advice thread, but I figured I'd start here.

I've been tasked with overhauling my company's digital document storage. I'm dealing with a lot of different types of files. Some date back as far as 2000. Others are from before a couple different things happened within the company, including a merger and another management company taking over things like marketing, HR, and Accounting. There is currently no clear set of document retention policies from our Board, so as part of the clean up, I've also been asked to create a proposal for what those should be.

A fair amount falls under just good practices for non-profits, like holding on to finances and resumes for specific lengths of time. However where I'm struggling is what to do with theatre specific things like old light plots and scenic designs. How do you manage those things? Are they something you get rid of on a rolling basis (2 years, 5 years, 10 year, etc)? Or are they something you keep permanently?
 

MNicolai

Well-Known Member
Fight Leukemia
Joined
Mar 30, 2008
Location
Sarasota, FL
Digital storage is cheap. Actual storage is...less so...and often flammable.

My application is different than a theater's but is fundamentally similar in that we have active projects, inactive-but-may-be-useful-to-keep-handy projects, and long forgotten projects. Digitally -- all projects, whether active or archived, are organized by project # in folders by calendar year. Active and recently inactive projects are kept in one volume on the network storage. Archived projects are on another. The archived volume is read-only for most users. If someone needs to bring a project out of the archives, they contact IT, but mostly what's in there is used only for reference and beyond a certain point nobody needs to have write access. In my case, each of those project folders include emails, correspondence, contracts, CAD files, PDF's, project photos, and basically everything related to each project except the HR/accounting/marketing type stuff -- which in terms of write access and workflow rightfully belongs elsewhere.

Having a somewhat standardized folder structure for each active project helps though obviously as organizations evolve, so do their documentation workflows and needs. Going into the archives is often the wild wild west -- pulling project info from 2006 often takes a lot more time because software applications for reading those files have changed and because it wasn't until the early 2010's in my company that everyone got more disciplined in keeping the folder structures better organized.

In terms of analog storage, both at my current and at my old company there was a tendency to hold onto paper documents just 'cuz. In both cases, that practice has mostly evaporated. Anything of historical, artistic, or creative significance is scanned and the paper copy may or may not be kept depending on what it is. Everything else was dumpstered. This was of particular importance at my old company because the paper archives were on shelves on a warehouse mezzanine above the offices. After 20-30 years, the paper archives had grown to a point where they presented a significant fire and structural hazard. It was time to stop being sentimental and liquidate. During the initial dumpster process, it's perfectly acceptable to hold onto stuff that's a question mark for another year or so to see if you still think it's valuable to hold onto.

Artistically, it's nice to keep analog copies of representative works or work products that were of significance to the development of the organization. Sometimes that history gets put on display somewhere -- other times it just serves as a milestone marker for people to see how the organization grew and developed over the years. There will be plenty of run-of-the-mill work products that do not meet that criteria. Also, in terms of artistic work product, if you know the designer involved, it's a nice courtesy to reach out and see if they want to hold onto anything before you toss it. Again though -- for your own sanity apply discretion. Not every piece of paper, sketch, or model is worth the time and effort to do that for. Really depends on what your organization has accumulated.

These days almost everything is digital and TB's of storage cost almost nothing so our overall retention policy is keep everything forever.

I can't talk about storage and not mention than any storage plan should include redundant backups that protect from the possibility of ransomware, malware, hardware failure, employee abuse, theft, accidental overwrites, fire, floods, and tornadoes. Having RAID partitions protects from disk failure but still has significant risk exposure from other areas. At a minimum you should have 3 copies of everything. Maybe the first two are a RAID volume or automated versioning backup of the primary partition, with the 3rd copy being off-site. Remember -- RAID by itself doesn't offer versioning or protection from the growing threat of ransomware. Don't be lulled into a false sense of security thinking that you can get by without the off-site backup, as my former employer did before he accidentally bounced the RAID enclosure off of the back of an equipment rack while pressing the power button, knocking it onto the floor -- thankfully all that data was recoverable though we suffered some downtime trying to resurrect it.

In the vein of redundancy, there are also some great options like Crashplan and Backblaze that offer relatively inexpensive cloud backups. I gave up Crashplan after I moved to RAID network drives, but certainly during college it saved my butt more than once to have file versioning to go back 4-5 versions and resurrect something I overwrote or an idea I discarded that I wanted to take a 2nd look at.
 
Last edited:

macsound

Well-Known Member
Joined
Jun 15, 2018
Location
San Francisco, CA
With community & regional theatre companies I've worked with, they always had an office and storage combined building. Like flex space in an office park. Most of storage was costumes and reusable props. The front office was big enough for a desk and couple of chairs and a conference room for meetings and usually the GM or exec producer.

Physical storage space for past works was essentially - as much as you had room for. There were definitely paper groundplans for shows from 20 years ago, but they were displayed. Anything in drawers was usually about 2x the thickness of the show's script, of which a photocopy would usually be saved because they were known to repeat popular shows and wanted to remember their notes. So essentially every single show they produced in 50 years could fit in 3-4 filing cabinets.

What happened over time, as they didn't want more cabinets but continued producing shows, is the slimming of the "bad" shows. Usually in the summer when shows weren't being produced, voulenteers would come help clean up. Props, costumes and office. If the show was a flop and they never planned on doing it again, anything specifically related to that show was trashed, all the paperwork, photos and videos archived on a DVD or 2 and the text files saved to the server.
So space was made by cleaning out, not necessarily by age, but by popularity. Even though the DVDs aren't technically archival, the assumption was made that they'd work for long enough in the future for anyone who wanted to dig back and by the time they went bad, no one would probably care. The files saved to the server (which backed up to tape at the time) was kept slim. Maybe 100MB per show.
 

Ben Stiegler

Well-Known Member
Joined
Aug 3, 2017
Location
Sf Bay Area
Digital storage is cheap. Actual storage is...less so...and often flammable.

My application is different than a theater's but is fundamentally similar in that we have active projects, inactive-but-may-be-useful-to-keep-handy projects, and long forgotten projects. Digitally -- all projects, whether active or archived, are organized by project # in folders by calendar year. Active and recently inactive projects are kept in one volume on the network storage. Archived projects are on another. The archived volume is read-only for most users. If someone needs to bring a project out of the archives, they contact IT, but mostly what's in there is used only for reference and beyond a certain point nobody needs to have write access. In my case, each of those project folders include emails, correspondence, contracts, CAD files, PDF's, project photos, and basically everything related to each project except the HR/accounting/marketing type stuff -- which in terms of write access and workflow rightfully belongs elsewhere.

Having a somewhat standardized folder structure for each active project helps though obviously as organizations evolve, so do their documentation workflows and needs. Going into the archives is often the wild wild west -- pulling project info from 2006 often takes a lot more time because software applications for reading those files have changed and because it wasn't until the early 2010's in my company that everyone got more disciplined in keeping the folder structures better organized.

In terms of analog storage, both at my current and at my old company there was a tendency to hold onto paper documents just 'cuz. In both cases, that practice has mostly evaporated. Anything of historical, artistic, or creative significance is scanned and the paper copy may or may not be kept depending on what it is. Everything else was dumpstered. This was of particular importance at my old company because the paper archives were on shelves on a warehouse mezzanine above the offices. After 20-30 years, the paper archives had grown to a point where they presented a significant fire and structural hazard. It was time to stop being sentimental and liquidate. During the initial dumpster process, it's perfectly acceptable to hold onto stuff that's a question mark for another year or so to see if you still think it's valuable to hold onto.

Artistically, it's nice to keep analog copies of representative works or work products that were of significance to the development of the organization. Sometimes that history gets put on display somewhere -- other times it just serves as a milestone marker for people to see how the organization grew and developed over the years. There will be plenty of run-of-the-mill work products that do not meet that criteria. Also, in terms of artistic work product, if you know the designer involved, it's a nice courtesy to reach out and see if they want to hold onto anything before you toss it. Again though -- for your own sanity apply discretion. Not every piece of paper, sketch, or model is worth the time and effort to do that for. Really depends on what your organization has accumulated.

These days almost everything is digital and TB's of storage cost almost nothing so our overall retention policy is keep everything forever.

I can't talk about storage and not mention than any storage plan should include redundant backups that protect from the possibility of ransomware, malware, hardware failure, employee abuse, theft, accidental overwrites, fire, floods, and tornadoes. Having RAID partitions protects from disk failure but still has significant risk exposure from other areas. At a minimum you should have 3 copies of everything. Maybe the first two are a RAID volume or automated versioning backup of the primary partition, with the 3rd copy being off-site. Remember -- RAID by itself doesn't offer versioning or protection from the growing threat of ransomware. Don't be lulled into a false sense of security thinking that you can get by without the off-site backup, as my former employer did before he accidentally bounced the RAID enclosure off of the back of an equipment rack while pressing the power button, knocking it onto the floor -- thankfully all that data was recoverable though we suffered some downtime trying to resurrect it.

In the vein of redundancy, there are also some great options like Crashplan and Backblaze that offer relatively inexpensive cloud backups. I gave up Crashplan after I moved to RAID network drives, but certainly during college it saved my butt more than once to have file versioning to go back 4-5 versions and resurrect something I overwrote or an idea I discarded that I wanted to take a 2nd look at.
Ditto on all that, but I use Backblaze (a whopping $75/year, unlimited storage, for 1 used with mapped network drives) and it has indeed saved my butt with versioning, corruption issues, and when i have oops-saved over a DSP design because I just knew that my change was correct but ... oops - bad things happened in unexpected ways and this was the best way to un-do.
 
  • Like
Reactions: RonHebbard

Ben Stiegler

Well-Known Member
Joined
Aug 3, 2017
Location
Sf Bay Area
On the wider topic ... I'm curious. If you as a designer are hired to mount a revival of a former show, are you interested in in the old light plot? sound effects and music? or do you prefer to start with a clean imagination?
 

TimMc

Well-Known Member
Premium Member
Joined
Feb 15, 2017
On the wider topic ... I'm curious. If you as a designer are hired to mount a revival of a former show, are you interested in in the old light plot? sound effects and music? or do you prefer to start with a clean imagination?
If I'm hired to do sound design for Seussical! (done it a couple of times) I may not start from scratch, but other than a character list and some notes about the score... it's relatively tabula rasa. Even for the same client... Sets, props, LX? Every director has a different approach to a show. Unless I were designing a show that was expected to generate future tech rentals, I'm not sure how much I'd save except for some portfolio examples.
 

TimMc

Well-Known Member
Premium Member
Joined
Feb 15, 2017
Curly, you've got me channeling my Inner Larry and Moe... ;)

I was secretary-treasurer for my IATSE Local for 8 years. Our document retention is determined by US Dept of Labor and IRS, and the International's constitution and bylaws. It's been a few years since I've thought about this, but I can't think of anything that exceeded 10 years. That said, the Local has 25 years or more of financials on computer and at least 10 years of paper records. The S-T's office has been on Apple machines from the beginning, so we have Time Machine and love it. When the money starts coming in I'm hoping to persuade the Board that Backblaze would be a smart way to handle off-site storage.

So with that, my advice is to ask the board if they have anyone who wants to looks over the treasure trove of Really Olde Paper before it gets recycled, shredded for pet litter... what ever you do with lots of dead trees. You mention a merger or acquisition... after 10 years? I doubt there's a legal requirement but you should have the board ask its attorney or authorize you to ask. It might be prudent to ask an attorney and accounting firm for document retention guidance or to propose policies for the board to consider. There might be documents of historic interest and the board may wish to form an archive committee to select governance, financial, and other records worthy of keeping in original form.

Artistic materials? Designs and other paperwork? Eh... if there's something about a particular show that makes it special (first show in "new" theater, etc) or the designer (who went on to Broadway fame)... or there is some kind of educational worth. Otherwise it's the theater version of Grandma's Attic.... maybe some cool stuff but mostly old junk. The Archive Committee can have more to do! :shhh:

Digitize whatever is relevant (play bill, posters, news paper clippings, set/LX/wardrobe design sketches) and archive the paper or 3D items where the medium is linked to the history. Is the Archive Committee meeting yet?
 

mbrown3039

Well-Known Member
Joined
Mar 20, 2018
Location
vegas, baby..!
Not sure if this really belongs here or in the General Advice thread, but I figured I'd start here.

I've been tasked with overhauling my company's digital document storage. I'm dealing with a lot of different types of files. Some date back as far as 2000. Others are from before a couple different things happened within the company, including a merger and another management company taking over things like marketing, HR, and Accounting. There is currently no clear set of document retention policies from our Board, so as part of the clean up, I've also been asked to create a proposal for what those should be.

A fair amount falls under just good practices for non-profits, like holding on to finances and resumes for specific lengths of time. However where I'm struggling is what to do with theatre specific things like old light plots and scenic designs. How do you manage those things? Are they something you get rid of on a rolling basis (2 years, 5 years, 10 year, etc)? Or are they something you keep permanently?
Start with the easy stuff - IIRC, IRS requires seven years (but, as an NP, 10 might be prudent); have HR do the research and make the recommendation on HR stuff; have Legal do the same (from experience, it seems that most lawyers want hard copies of anything current and at least the signature page of anything not current). Drawings, plots and such? Unless submitted for permit (in which case, keep the wet copy of anything still relevant/in place), it's all scan-n-recycle.

BTW, old designs/system drawings can be used as wallpaper, framed "tech art," or donated to high school/community college drafting courses as teaching aides. m
 
  • Like
Reactions: RonHebbard

Crisp image

Well-Known Member
Joined
Jun 18, 2017
Location
Eastern Victoria Australia
Where I work (Age care facility)we keep paper documents for 7 years. Things like medical records, Financial and employee records and reports for compliance.
Digital stuff is backed up onsite every hour to an external NAS and then offsite every 24hrs with version control. Saved us a couple of times.
As for the PAC I work at it is run by local government so I guess they have rules for their systems but I don't know about them and will never access them either.

Regards

Geoff
 
  • Like
Reactions: RonHebbard

DrewE

Well-Known Member
Joined
Mar 18, 2019
Location
Vermont
For digital backups, make sure you regularly test actually getting data back, not just that it (seems) to go to wherever it is being sent. That may sound rather obvious, but it's easy to overlook or get complacent. A backup that you can't in fact restore is no good, and arguably worse than no backup at all in as much as you get a false sense of security. Here's an anecdote I'm personally familiar with (though, thankfully, not much affected by).

When I was a student at RIT (the Rochester Institute of Technology), they had a rather nasty and somewhat embarrassing problem with the main student UNIX system. The main disks for the system were a RAID array, and one of the drives in the array failed. When they detected this failed drive, they of course obtained a replacement, put it in, and issued the commands to populate the new drive with the appropriate data based on the other drives, as expected. Unfortunately, the documentation or software or something for the array had errors and instead of collecting the data from the good drives and filling the new drive, the operation actually worked in reverse and took the blank data from the new drive and spread it all around to the existing drives, with the result that the data was all destroyed.

At this point, the IT people naturally turned to their tape backups, and made the discovery that the part of the file system with all the user accounts and everyone's data was not being backed up because the main user directory had too many subdirectories (one for each user's account) for whatever backup program they were using. Apparently deep in the logs somewhere it did put a diagnostic message noting that it was skipping the directory for this reason, but nevertheless still claimed a successful backup operation when it was done.

Needless to say, this whole boondoggle caused quite a few people no small amount of difficulty. The producer of the RAID array was eventually able, after several months, to resurrect much of the data, as I recall. The IT group also revised their practices to better verify backups, and moved to a different backup software.