[Contents]   [Back]   [Prev]   [Up]   [Next]   [Forward]  


SCM Interface

DBSCM is a disk based, sorted associative array package (WB) integrated into the Scheme implementation SCM. These associative arrays consist of keys which are strings of length less than 256 bytes and values which are strings of length less than 64770.B. Associative arrays can be used to form a MUMPS style database which can be used to implement standard record structures without auxiliary files (see example in example.scm).

The WB implementation (compiled) adds about 66 kilobytes to the size of SCM.

SCM Status Codes

Function: err? x
Return x if a valid error code (-1 ... MAXERR); else #f.

Function: success? x
Not err?.

constant: success
Successful execution (0).

Negative integers are used for errors according to increasingly severity, as follows:

constant: notpres
Successful execution, no data present or no change made

constant: terminated
Failure, no damage, caller can retry operation

constant: retryerr
Failure, no damage, caller can retry operation

constant: argerr
Failure, no damage, call was in error

constant: noroom
Failure, no damage, out of room in file.

constant: typerr
Failure, file or object was not of correct type.

constant: ioerr
I/O error, DB may be damaged.

constant: strangerr
Internal error, DB may be damaged.

constant: unkerr
Placeholder code.

constant: maxerr
All error codes are between 0 and maxerr.

SCM Segments

The block-size of a segment (given in call to make-seg) is a parameter crucial for performance; balancing CPU time traversing blocks with file-system latency. block-size should be an integer multiple of the file-system block size.

In the 1990s our nominal block-size was 2.kiB; now it should probably be 4.kiB, 8.kiB, or 16.kiB.

Scheme Procedure: init-wb max-num-ents max-num-buks max-blk-size

Initializes the WB system. Max-blk-size determines the size of the disk cache buffers. max-num-ents is the number of disk cache buffers to be allocated. (* max-num-ents Max-blk-size) should be less than the size of RAM on your computer. If not all max-num-ents cannot be allocated (by malloc) then WB can still run properly. max-num-buks is the number of hash buckets for the disk cache. It should be of approximately the same size as or larger than max-num-ents. The number of buffers actually allocated is returned if successful; a status code is returned otherwise.

If open-seg or make-seg is called when init-wb is not called, then WB will be initialized as though init-wb was called with the parameters 75, 150, and 2048.

Scheme Procedure: final-wb

Frees all memory used by the WB system. All segments will be closed.

Scheme Procedure: open-seg seg filename mode

Seg should be a non-negative number. Opens the database seg in filename and returns seg if successful, a status code otherwise. The database seg will be read-only if the mode argument is 0. It will be read-write if the mode argument is 2.

Scheme Procedure: close-seg seg hammer

Closes database segment seg and the file containing it. If hammer is #f then if there are any problems freeing buffers then the close is aborted. A status code is returned.

Scheme Procedure: make-seg seg filename block-size

The integer seg must specify an unused segment. The integer block-size specifies the size of B-tree blocks. block-size should be an integer multiple of the file-system block size. Nominal value is 4096.

make-seg creates a new empty database seg of name filename. A status code is returned. Call open-seg to use the new database.

SCM B-Trees

The write-control-bits argument (WCB) to these functions controls the latency of updates to the file after various operations. These bits are defined as follows:
value Name Meaning
1 WCB-SAP save block after PUTs
2 WCB-SAR save block after REMOVEs
4 WCB-SAC force block save after cached block changes
8 WCB-FAC flush buffer entirely after cached block changes (not currently implemented)

Scheme Procedure: create-bt seg typ wcb

Creates a new root block in seg seg of type typ and returns a bt-handle open to it if successful; otherwise #f. This would typically be used to create a temporary b-tree which should be reclaimed by check if system crashes.

Scheme Procedure: open-bt seg blknum wcb

Returns a bt-handle open to seg number seg, block number blknum if successful; otherwise #f. If no such block exists or is not a root block a status code is returned.

For create-db and open-db, the implicit WCB argument is the combination of `WCB-SAP' and `WCB-SAR'.

Scheme Procedure: create-db seg typ name

Returns a B-tree whose name has been entered in the root directory if successful; otherwise #f.

typ should be either

B-trees with typ #\D which are pointed to by special entries in the root block (1) protect all their special entries from garbage collection by the check program. #\T is for regular (data) arrays.

Scheme Procedure: open-db seg name

Returns the B-tree whose name has been entered in the root directory or #f if not found.

Dirty block buffers can also be flushed to disk by calls to flush-ents. flush-ents can be called at any time after WB is initialized, even by an asynchronous background process.

Scheme Procedure: flush-ents attempts k

k is the number of dirty block buffers to write to disk; attempts is the number of times to try. Note that blocks in any segment may be written by flush-ents. flush-ents returns the number of blocks written.

Block numbers are stored in the directory as four-byte integers. In order to make WB files portable between big-endian and little-endian computers, all conversions of four-byte pointers should be done by str2long and long2str!.

Scheme Procedure: str2long string index

Converts the 4 bytes in string starting at index into an unsigned integer and returns it.

Scheme Procedure: long2str! string index integer

Stores integer into 4 bytes of string starting at index.

SCM Record Operations

Scheme Procedure: bt:get han key

han is a handle to an open bt. key is a string less than 255.B in length.

bt:get returns a string of the value associated with key in the bt which han is open to. bt:get returns #f if key is not associated in the bt.

Scheme Procedure: bt:next han key

han is a handle to an open bt. key is a string less than 255.B in length.

bt:next returns the next key in bt han or #f if none.

Scheme Procedure: bt:prev han key

han is a handle to an open bt. key is a string less than 255.B in length.

bt:prev returns the previous key in bt han or #f if none.

Scheme Procedure: bt:put! han key val

han is a handle to an open, mutable bt. key and val are strings less than 255.B in length.

bt:put! associates key with val in the bt han. A status code is returned.

Scheme Procedure: bt:rem! han key

han is a handle to an open, mutable bt. key is a string less than 255.B in length.

bt:rem! removes key and it's associated value from bt han.

The value strings bt:get and bt:put! work with are limited to 255.B in length. db:get and db:put! work with value strings up to 64770.B in length.

Scheme Procedure: db:get han key

han is a handle to an open bt. key is a string less than 255.B in length.

db:get returns a string (up to 64770.B long) of the value associated with key in the bt which han is open to. Returns #f if key is not in the bt.

Scheme Procedure: db:put! han key val

han is a handle to an open, mutable bt. key is a string less than 255.B in length. val is a string less than 64770.B in length.

db:put! associates key with val in the bt han. A status code is returned.

SCM Mutual Exclusion

These 2 calls can be used for locking and synchronizing processes.

Scheme Procedure: bt:put han key val

Associates key with val in the bt han only if key was previously empty. Returns #t for success, #f for failure.

Scheme Procedure: bt:rem han key

Removes key and it's associated value from bt han only if key is present. Returns key's value for success, #f for failure (not present).

SCM Multiple Operations

Scheme Procedure: bt:rem* han key1 key2

Removes keys (and their associated values) between (including) key1 and (not including) key2 from bt han. A status code is returned.

Scheme Procedure: bt:scan han op key1 key2 func blklimit

Applies procedure func to a range of keys (and values) and either counts, modifies, or deletes those associations. A list of a status code, the count of records processed, and a new value for key1 is returned.

If op is -1, indicated keys will be deleted; If op is 0, indicated keys will be counted; If op is 1, the value of each key in the range will be changed to be the string returned by func.

Func is called with 2 string arguments, the key and the value. If op is 1 and func returns #f then no change will be made. If op is 1 and func returns a string then that string will replace the value for that key. For the other (count, delete) modes, func should return #f or #t. If func is #t, the association will be counted (and possibly deleted).

Keys from key1 (inclusive) up to key2 (exclusive) are scanned. If blklimit is -1 then the entire range is scanned; otherwise only as many blocks (internal database structures) as specified by blklimit are scanned. The scan can be restarted by using the returned information. For instance, the following expression counts the number of keys between "3" and "4" one block at a time and returns a list of the number of keys and the number of blocks.

(do ((res (bt:scan current-bt 0 "3" "4" (lambda (k v) #t) 1)
          (bt:scan current-bt 0 (caddr res) "4" (lambda (k v) #t) 1))
     (blks 0 (+ 1 blks))
     (cnt 0 (+ cnt (cadr res))))
    ((not (= -1 (car res))) (list (+ cnt (cadr res)) (+ 1 blks))))

It is good to specify a positive blklimit when dealing with large scans. More details on the operation of scan can be found in `scan.scm'.

SCM Diagnostics

The value returned by the following routines is unspecified.

Scheme Procedure: check-access

Checks that buffers and locks are in a consistent state and fixes them if WB routines have been interrupted.

Scheme Procedure: clear-stats

Resets all the collected statistics to 0.

Scheme Procedure: stats

Prints a summary of operational statistics since the last clear-stats or cstats.

Scheme Procedure: cstats

Prints a summary of operational statistics since the last clear-stats or cstats, then calls clear-stats.

Scheme Procedure: show-buffers

Prints a table of the status of all disk cache buffers.


[Contents]   [Back]   [Prev]   [Up]   [Next]   [Forward]