2.0 > SYSTEMS TO SUPPORT MANAGEMENT OF THE DIGITAL OUTPUT OF DIGITISATION AS RECORDS MUST BE IN PLACE

The purpose of the digitisation will determine the functionality and system requirements necessary to manage the digital outputs.

Prior to digitisation, consideration of third party copyright or other constraints inherent in the record SHOULD be resolved.

Before the final version is reached, the digitising process could create a number of versions of a digitised record – for example, a raw file, one which has been enhanced to maximise resemblance to the original, or one to which quality assurance processes have been applied. Each digitisation process will specify the parameters of acceptable enhancements (see Requirement 2.1). The final output of the digitisation process MUST be regarded as the record for incorporation into the organisation's recordkeeping framework.

Many public offices and local authorities use electronic document and records management systems as their image control systems. Such systems SHOULD be assessed for functionality against Archives New Zealand's Electronic Recordkeeping Systems Standard (see the Continuum Resource Kit).

Where the digital image is to be used as a record in current or continuing business, the system governing the business process which will use the image SHOULD be integrated with records control systems. This ensures that the digital image will inherit the business controls and metadata associated with the business process, will be lodged within its business context and will have its authenticity enhanced by integration with the normal business systems.

Digitisation projects are often undertaken as a preservation measure to extend the life of fragile original records or to increase access by users to the informational content of records. Where the results of such projects are not immediately linked to a pre-existing business system, consideration of the software to be used to manage the images is necessary. Where the digital images represent a considerable investment of time and resources, serious consideration SHOULD be given to purchasing a management system to ensure appropriate control of processes such as identification, indexing, classification, security and access controls, rights management and preservation.

Back to top

2.1 > SOURCE RECORD PREPARATION GUIDELINES MUST BE DOCUMENTED AND IMPLEMENTED

The aim of producing digital images is to reproduce the original as faithfully as possible so that the digital image can act in place of the original, where it is required to act as evidence of business activities.

Source record preparation guidelines are not restricted to, but SHOULD include:

• An assessment of the source record's capability to sustain a digitisation process (e.g. paper quality, creasing, stapling, condition of microform jackets, attributes of the informational content – e.g. graphics);
• Methods of digitising non-standard source records (e.g. by digitising a photocopy of records on fragile or thin paper; creating a standard-sized document by using photocopy enlargement or reduction; enclosing fragile originals in plastic wallets or using specialty devices such as overhead scanners) and quality checks to ensure against data loss in any such processes;
• Methods for dealing with source records containing handwritten annotations, marginalia, white opaque paint or highlighted areas;
• Methods of distinguishing between original source records and photocopies;
• Guidance on what types of material need not be digitised as they are of only ephemeral or short-term value (guidance on this is available from Archives New Zealand's General Disposal Authority 3: General Housekeeping Records);
• Physical preparation for digitisation (e.g. careful staple removal, alignment of single pages, batching of like documents – size, technical settings, shared indexing fields);
• Processes for assigning links between associated documents to be regarded as a single item, so that the digitised image can faithfully represent the source record (e.g. a document and a 'post it' note attached; a document enclosing an attachment, etc.);
• Processes for assigning links between the source record and the digitised copy. Such links will usually be documented using identification protocols. In some applications barcode technology could be used to link paper and digitised versions;
• Procedures to enable checking and verification that all target source records have been included in the digitisation process;
• Principles governing the assembly of batches or groups of source records suitable for digitisation at the same time (e.g. size, colour, date order, document formats, orientation – portrait or landscape, single or double sided, etc.).

Back to top

2.2 > ALL DIGITISED IMAGES MUST BE ASSIGNED METADATA TO DOCUMENT DIGITISING PROCESSES AND TO SUPPORT ONGOING BUSINESS PROCESSES³

Metadata attributed to, or associated with, images is an essential component in the management and retrieval of images. Wherever possible metadata SHOULD be inherited from existing systems.

Two types of metadata MUST be captured:

• Metadata specific to the particular image and the imaging process; and
• Metadata about the record, the business being transacted and the agents associated with the business.

The majority of this metadata can be automatically sourced from the software and hardware used to manage the digitising process. The manual attribution or application of metadata SHOULD be minimised.

Metadata can be embedded with the resource in header information, or can be managed in a separate system, or both, but in either case there has to be a direct relationship or association between them. That is, while metadata may reside in a separate system, it has to link directly to the resources. Metadata can also be encapsulated within the image format. Choices for storage of metadata SHOULD be made according to the principles outlined in ISO 23081 – 1: 2006, Information and documentation – Records management processes – Metadata for records.

Image-level metadata

Image-level metadata SHOULD be generated automatically at the point of digital capture direct from the digitisation equipment and SHOULD avoid manually-assigned data entry wherever possible.

In addition to the metadata inherited from recordkeeping capture and processes, or from indexing and searching metadata, image-level metadata MUST include:

• Unique digital image identifier;
• Date and time of digitisation;
• The name of the agent associated with the digitisation process (e.g. name of the outsourced bureau or name of the in-house operator);
• Capture device (hardware and software);
• Calibration settings (resolution, colour, dimensions, etc.); and
• Date of last calibration.

Additional image-level metadata MAY be assigned at the discretion of the public office or local authority.

Recommendations on naming protocols for digital images and directories are included in Appendix 6: File-naming Metadata Recommendations.

Business-process digitisation

Wherever possible the metadata controlling the business process and the recordkeeping functions associated with the business process SHOULD govern, and be inherited by, the specific digital image. This metadata MUST be incorporated into the organisation's electronic recordkeeping framework and be consistent with ISO 23081 – 1: 2006, Information and documentation – Records management processes – Metadata for records. Additional metadata describing the process of digitisation and specific characteristics of the digitised image MUST be included, as outlined above.

Digitisation projects

Where access to content is the primary driver, more intensive attention to providing additional indexing and search entry points is appropriate. The images can be managed as individual items, rather than as context-linked records, particularly if intended for web access by external users. Relevant elements from the New Zealand Government Locator Service (NZGLS) metadata SHOULD be associated with each image.

Back to top

2.3 > QUALITY ASSURANCE PROCEDURES MUST BE DEFINED, DOCUMENTED AND IMPLEMENTED

Quality assurance to ensure the digital copy of the source record is a true and accurate copy is critical to being able to assert that the records possess integrity and are authentic.

Such quality assurance procedures SHOULD be documented and built into the ongoing operation of the digitising process, not considered simply a check on output only.

Quality assurance procedures SHOULD, at minimum, address the following issues:

• Any acceptable variations from normal procedures;
• Scanner operation quality control;
• Verification that digital output matches the quantity of original record input;
• Extent and frequency of sampling of digitised images;
• Criteria for checking image quality;
• Frequency and criteria for checks on metadata;
• Processes for re-digitising; and
• Operator training.

Quality checking MUST be completed before the digitised images are accepted into a business process, or as a master copy in the case of digitisation projects. Quality checking MUST be complete before the destruction of the source records is considered.

The results of quality assurance processes and quality checks MUST be documented.

A review of quality procedures for digitising SHOULD be undertaken regularly to ensure that the procedures continue to meet the business requirements.

Appropriate training SHOULD be provided to all staff who create, manage or work with digitised records. Documentation on the level and the frequency of training provided to those staff involved with digitisation SHOULD be created and maintained.

For further information on quality assurance, see Appendix 7: Quality Assurance Recommendations, and Appendix 8: Recommended Staff Skill Sets.

Back to top

2.4 > STORAGE MEDIA AND BACK-UP PROCEDURES MUST BE DEFINED, DOCUMENTED AND IMPLEMENTED

Maintaining technology-dependent records in networked storage is currently the storage strategy most likely to ensure the continuing accessibility of such records over time. However, digitised records can occupy significant storage space depending on the quality, resolution and compression ratios employed. Strategies for storage MAY include:

• A dedicated server or other digitised record storage solution;
• Writing the digitised records to magnetic tape;
• Writing the digitised records to WORM (write once, read many) storage media (e.g. a CD or DVD).

The requirements for storage include:

• That the digitised records MUST be unalterable in all storage media;
• That security and access controls for storage media MUST be capable of detecting and logging unauthorised attempts at access;
• That the retrieval times implicit in off-line storage SHOULD be acceptable for the business being conducted;
• Wherever possible digitised records sharing similar retention periods SHOULD be co-located to enable execution of destruction processes as required.

All digitised records, and their associated metadata, MUST be included in the organisation's back-up regime. Back-up procedures are designed to provide sufficient up-to-date copies of business records to be used in the event of loss or corruption of all or part of the data.

Back-up regimes SHOULD be documented and back-up copies maintained to a level of security that will ensure the authenticity of the records used in recovery situations.

All system failures MUST be documented, and use of the back-up copies for restoration purposes MUST be accompanied by verification testing to ensure the integrity of the restored records.

Information technology professionals often use the term 'archiving' to describe back-up regimes. For recordkeeping purposes conducting back-ups does not constitute an archiving or preservation strategy, it is a business continuity or disaster recovery precaution.

---

3. Please note: This section has been designed not to be prescriptive, but to identify areas of critical importance for maintaining the authenticity and integrity of the digitised record. This enables organisations to adapt specific elements as required and to maximise the inheritance of data values from existing systems and equipment. It is designed to maximise automatic capture of metadata, not manual attribution.