Converts MARC-8 encoded records to UTF-8 encoding

Converts MARC21 to MARCXML21

Select the unique record number field (common choices are '035' or '001'; check your records as you might be surprised how a supposedly unique field actually has duplicates, though marc2bre.pl will select a unique identifier for subsequent duplicates)

Extracts certain pertinent fields for indexing and display purposes (along with the complete MARCXML21 record)

Sets the ID number of the first record from this batch to be imported into the biblio.record_entry table (hint - run the following SQL to determine what this number should be to avoid conflicts:

psql -U postgres evergreen
 # SELECT MAX(id)+1 FROM biblio.record_entry;

If you are processing multiple sets of MARC records with marc2bre.pl before loading the records into the database, you will need to keep track of the starting ID number for each subsequent batch of records that you are importing. For example, if you are processing three files of MARC records with 10000 records each into a clean database, you would use –startid 1, –startid 10001, and –startid 20001 parameters for each respective file.

Ignore “trash” fields that you do not want to retain in Evergreen

If you use marc2bre.pl to convert your MARC records from the MARC-8 encoding to the UTF-8 encoding, it relies on the MARC::Charset Perl module to complete the conversion. When importing a large set of items, you can speed up the process by using a utility like marc4j or marcdumper to convert the records to MARC21XML and UTF-8 before running them through marc2bre.pl with the –marctype=XML flag to tell marc2bre.pl that the records are already in MARC21XML format with the UTF-8 encoding. If you take this approach, due to a current limitation of MARC::File::XML you have to do a horrible thing and ensure that there are no namespace prefixes in front of the element names. marc2bre.pl cannot parse the following example:

<?xml version="1.0" encoding="UTF-8" ?>
<marc:collection xmlns:marc="http://www.loc.gov/MARC21/slim" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:schemaLocation="http://www.loc.gov/MARC/slim 
http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd">
  <marc:record>
    <marc:leader>00677nam a2200193 a 4500</marc:leader>
    <marc:controlfield tag="001">H01-0000844</marc:controlfield>
    <marc:controlfield tag="007">t </marc:controlfield>
    <marc:controlfield tag="008">060420s1950    xx            000 u fre d</marc:controlfield>
    <marc:datafield tag="040" ind1=" " ind2=" ">
      <marc:subfield code="a">CaOHCU</marc:subfield>
      <marc:subfield code="b">fre</marc:subfield>
    </marc:datafield>
...
;

But marc2bre.pl can parse the same example with the namespace prefixes removed:

<?xml version="1.0" encoding="UTF-8" ?>
<collection xmlns:marc="http://www.loc.gov/MARC21/slim" 
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:schemaLocation="http://www.loc.gov/MARC/slim 
http://www.loc.gov/standards/marcxml/schema/MARC21slim.xsd">
  <record>
    <leader>00677nam a2200193 a 4500</leader>
    <controlfield tag="001">H01-0000844</controlfield>
    <controlfield tag="007">t </controlfield>
    <controlfield tag="008">060420s1950    xx            000 u fre d</controlfield>
    <datafield tag="040" ind1=" " ind2=" ">
      <subfield code="a">CaOHCU</subfield>
      <subfield code="b">fre</subfield>
    </datafield>
...
;