Data Export

Moving data out of a system can mean a nightly file to a partner company, a feed to another internal application, or an extract a customer pulls on request. When the source system was not set up to share data this way, the export is built around it, along with the business rules about what can be shared and with whom.

Exports can run on a schedule, on demand, or when a specific event happens. The harder question is how both sides know a delivery worked. A confirmation mechanism can combine some of these:

• a control record with row counts and totals, so the receiver can tell a partial file from a complete one
• a checksum, such as SHA-256, for each delivered file
• a stable batch identifier that the receiver records and checks before processing a resend, so the same batch is not loaded twice
• an acknowledgment from the receiving system, with an alert to your staff when one does not arrive

What a resend does is agreed with the receiver. Some feeds replace an earlier batch; an append-only or accounting feed may need a correction sent as new adjusting records instead of overwriting data it has already accepted.

Bringing data in from partners is the mirror of this work; see Data Import.