Transaction Backup - Revision history

Barb at 22:31, 2 February 2010

2010-02-02T22:31:33Z

← Older revision		Revision as of 22:31, 2 February 2010
Line 78:		Line 78:
	#Remember this transaction number for the ''next'' checkpoint operation (2223).		#Remember this transaction number for the ''next'' checkpoint operation (2223).
	#Playback all transactions from 1111 to 2222 to the backup folder.		#Playback all transactions from 1111 to 2222 to the backup folder.
-	#*If the destination file does not exist for the transaction, ~~copy the file from the active folder~~. ''There may be some danger in this.''	+	#*If the destination file does not exist for the transaction, simply ignore it. ''There may be some danger in this.''
	#Make a Checkpoint file by compressing the backup folder. Name the compressed file using the last transaction number it contains (2222).		#Make a Checkpoint file by compressing the backup folder. Name the compressed file using the last transaction number it contains (2222).

Jim: /* Recording Transactions */ attempted clarification

2010-02-02T16:39:44Z

Recording Transactions: attempted clarification

← Older revision		Revision as of 16:39, 2 February 2010
Line 28:		Line 28:
	There are a finite number of these packets that the file server receives. These "transactions" will be written to a file. Each transaction will have a serial number which just increments each time a transaction is written. The transaction file will be "cycled" every so often. That is, a new transaction file will be created whose name contains the first transaction number in it. Cycling will be triggered by an action such as a Backup or Snapshot, some time increment, or some number of transactions. This will produce potentially many transaction files, but will allow them to be transmitted from server to server relatively easily.		There are a finite number of these packets that the file server receives. These "transactions" will be written to a file. Each transaction will have a serial number which just increments each time a transaction is written. The transaction file will be "cycled" every so often. That is, a new transaction file will be created whose name contains the first transaction number in it. Cycling will be triggered by an action such as a Backup or Snapshot, some time increment, or some number of transactions. This will produce potentially many transaction files, but will allow them to be transmitted from server to server relatively easily.

-		+	*Transactions are atomic, in that every transaction will contain sufficient information so that the file in question will be changed correctly using only the information in the transaction.
-	*A Transaction File will be a Binary file with variable length records.	+	*A Transaction File will be a Binary file with variable length records.
	*Each record in a Transaction file will contain:		*Each record in a Transaction file will contain:
-	*# Length of entire record
	*# Length of entire record		*# Length of entire record
	*# Length of this header		*# Length of this header
Line 44:		Line 43:
	*# Path to file - up to 260 bytes (If Keyed File, Points to the data file)		*# Path to file - up to 260 bytes (If Keyed File, Points to the data file)
	*# Packed Data -- Depends on operation -- up to 8194 bytes -- could contain the Key		*# Packed Data -- Depends on operation -- up to 8194 bytes -- could contain the Key
		+
		+	* What about record location information? Is there sufficient info in the packed data to position the file correctly before the transaction is executed?
		+
		+	* What about DOS calls?

	* What about encryption? Is this needed? If so, at what point in the process?		* What about encryption? Is this needed? If so, at what point in the process?

Jim: /* Recording Transactions */

2010-02-01T18:32:27Z

Recording Transactions

← Older revision		Revision as of 18:32, 1 February 2010
Line 31:		Line 31:
	*A Transaction File will be a Binary file with variable length records.		*A Transaction File will be a Binary file with variable length records.
	*Each record in a Transaction file will contain:		*Each record in a Transaction file will contain:
-
	*# Length of entire record		*# Length of entire record
	*# Length of entire record		*# Length of entire record

Jim: /* Recording Transactions */ changed from ascii to binary

2010-02-01T18:31:46Z

Recording Transactions: changed from ascii to binary

← Older revision		Revision as of 18:31, 1 February 2010
Line 29:		Line 29:


-	*A Transaction File will be a ~~Text~~ file with variable length records.	+	*A Transaction File will be a Binary file with variable length records.
	*Each record in a Transaction file will contain:		*Each record in a Transaction file will contain:
-	*# ~~Header~~ Length ~~- 4 bytes. Decimal number~~ of ~~bytes before the raw packed data. (includes these 4 bytes)~~	+
-	*# ~~Transaction~~ number - up to 8 bytes ~~(Decimal)~~	+	*# Length of entire record
-	*# Time ~~Stamp - Local date and time of the~~ transaction~~. YYYYMMDDhhmmss~~	+	*# Length of entire record
-	*# ~~Partition Number~~ - up to 4 bytes ~~(Decimal)~~	+	*# Length of this header
-	*# ~~Operation - up to 2 bytes (Hex)~~	+	*# Packed Data Length
-	*# ~~Node Name~~ -- up to 16 bytes ~~(Quoted)~~	+	*# Unpacked Data Length
-	*# ~~Program Name~~ - up to 8 bytes ~~(Quoted)~~	+	*# This transaction number - up to 8 bytes
-	*# ~~File~~ Path - up to 260 bytes (If Keyed File, Points to the data file~~) (Quoted)~~	+	*# Time this transaction was recorded
-	*# Unpacked Data Length - 4 bytes (Decimal)	+	*# Id of task - up to 4 bytes
-	*# Packed Data Length - 4 bytes (Decimal)	+	*# Function performed
-	*# Packed Data -- Depends on operation -- up to 8194 bytes -- could contain the Key	+	*# Client node -- up to 16 bytes
-	* All Header fields will be separated with commas.	+	*# Running program - up to 8 bytes
-	* Transaction records end with CR/LF.	+	*# Path to file - up to 260 bytes (If Keyed File, Points to the data file)
		+	*# Packed Data -- Depends on operation -- up to 8194 bytes -- could contain the Key
		+
	* What about encryption? Is this needed? If so, at what point in the process?		* What about encryption? Is this needed? If so, at what point in the process?

Jim: Added causes for transaction backup

2010-02-01T18:19:55Z

Added causes for transaction backup

← Older revision		Revision as of 18:19, 1 February 2010
Line 12:		Line 12:
	**Create a Key		**Create a Key
	**Erase a File		**Erase a File
		+	**Clear a File
	**Rename a File		**Rename a File
	**Delete a record		**Delete a record
	**Write a record. (includes all variants such as write, rewrite, update, insert, etc.)		**Write a record. (includes all variants such as write, rewrite, update, insert, etc.)
		+	**Windows -- Create Directory
		+	**Windows -- Create File
		+	**Windows -- Move File
	*'''Backup''' -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.		*'''Backup''' -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.
	*'''Snapshot''' -- a process whereby transactions are applied to the backup repository, creating a checkpoint.		*'''Snapshot''' -- a process whereby transactions are applied to the backup repository, creating a checkpoint.

Jim: minor

2009-09-16T17:26:25Z

minor

← Older revision		Revision as of 17:26, 16 September 2009
Line 5:		Line 5:
	=== First, some definitions. ===		=== First, some definitions. ===

-	*'''Live Data''' -- all of the directories and files that the file server ~~modifies~~. This is comprised of all directories called for by any user executing a "DirAdd" call.	+	*'''Live Data''' -- all of the directories and files that the file server could modify. This is comprised of all directories called for by any user executing a "DirAdd" call.
	*'''Backup repository''' -- a place, usually a single folder, containing a faithful copy of the live data at some given point in time.		*'''Backup repository''' -- a place, usually a single folder, containing a faithful copy of the live data at some given point in time.
	*'''Checkpoint''' -- that point in time where all data in the backup repository matches the live data.		*'''Checkpoint''' -- that point in time where all data in the backup repository matches the live data.

Jim: minor

2009-09-15T23:22:39Z

minor

← Older revision		Revision as of 23:22, 15 September 2009
Line 17:		Line 17:
	*'''Backup''' -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.		*'''Backup''' -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.
	*'''Snapshot''' -- a process whereby transactions are applied to the backup repository, creating a checkpoint.		*'''Snapshot''' -- a process whereby transactions are applied to the backup repository, creating a checkpoint.
-	*'''Rollback''' -- a process that copies all of the files comprising a ~~checkpoint~~ back to ~~live data~~ and possibly applies transactions to it up to some arbitrary point, reestablishing the ~~live data~~ at some past point in time.	+	*'''Rollback''' -- a process that copies all of the files comprising a Checkpoint back to Live Data and possibly applies transactions to it up to some arbitrary point, reestablishing the Live Data at some past point in time.

	==Recording Transactions==		==Recording Transactions==

Jim: minor

2009-09-15T23:21:19Z

minor

← Older revision		Revision as of 23:21, 15 September 2009
Line 10:		Line 10:
	*'''Transaction''' -- any file activity that changes a file.		*'''Transaction''' -- any file activity that changes a file.
	**Create a File		**Create a File
-	**Create a Key?	+	**Create a Key
	**Erase a File		**Erase a File
	**Rename a File		**Rename a File
Line 17:		Line 17:
	*'''Backup''' -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.		*'''Backup''' -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.
	*'''Snapshot''' -- a process whereby transactions are applied to the backup repository, creating a checkpoint.		*'''Snapshot''' -- a process whereby transactions are applied to the backup repository, creating a checkpoint.
-	*'''Rollback''' -- a process that copies all of the files comprising a checkpoint back to live data and possibly applies transactions to it up to some arbitrary point, reestablishing the live data at some past point in time.	+	*'''Rollback''' -- a process that copies all of the files comprising a checkpoint back to live data and possibly applies transactions to it up to some arbitrary point, reestablishing the live data at some past point in time.
		+
	==Recording Transactions==		==Recording Transactions==

Jim: minor

2009-09-15T23:20:16Z

minor

← Older revision		Revision as of 23:20, 15 September 2009
Line 5:		Line 5:
	=== First, some definitions. ===		=== First, some definitions. ===

-	*Live Data -- all of the directories and files that the file server modifies. This is comprised of all directories called for by any user executing a "DirAdd" call.	+	*'''Live Data''' -- all of the directories and files that the file server modifies. This is comprised of all directories called for by any user executing a "DirAdd" call.
-	*Backup repository -- a place, usually a single folder, containing a faithful copy of the live data at some given point in time.	+	*'''Backup repository''' -- a place, usually a single folder, containing a faithful copy of the live data at some given point in time.
-	*Checkpoint -- that point in time where all data in the backup repository matches the live data.	+	*'''Checkpoint''' -- that point in time where all data in the backup repository matches the live data.
-	*Transaction -- any file activity that changes a file.	+	*'''Transaction''' -- any file activity that changes a file.
	**Create a File		**Create a File
	**Create a Key?		**Create a Key?
Line 15:		Line 15:
	**Delete a record		**Delete a record
	**Write a record. (includes all variants such as write, rewrite, update, insert, etc.)		**Write a record. (includes all variants such as write, rewrite, update, insert, etc.)
-	*Backup -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.	+	*'''Backup''' -- a process that makes an exact copy of the live data into the backup repository, creating a checkpoint.
-	*Snapshot -- a process whereby transactions are applied to the backup repository, creating a checkpoint.	+	*'''Snapshot''' -- a process whereby transactions are applied to the backup repository, creating a checkpoint.
-	*Rollback -- a process that copies all of the files comprising a checkpoint back to live data and possibly applies transactions to it up to some arbitrary point, reestablishing the live data at some past point in time.	+	*'''Rollback''' -- a process that copies all of the files comprising a checkpoint back to live data and possibly applies transactions to it up to some arbitrary point, reestablishing the live data at some past point in time.
	==Recording Transactions==		==Recording Transactions==

Jim: minor

2009-09-15T23:18:43Z

minor

← Older revision		Revision as of 23:18, 15 September 2009
Line 76:		Line 76:

	==Rollback==		==Rollback==
-	At any time, the user can "rollback" the whole system by restoring the Live Data up to a ~~checkpoint~~ and then possibly playing transactions up to a desired point in time. This could be on a completely different system than the main server. This would facilitate migration to a new server, making a test system, or whatever.	+	At any time, the user can "rollback" the whole system by restoring the Live Data up to a Checkpoint and then possibly playing transactions up to a desired point in time. This could be on a completely different system than the main server. This would facilitate migration to a new server, making a test system, or whatever.

	==Added Benefits==		==Added Benefits==