org.abora.gold.snarf
Class SnarfPacker

java.lang.Object
  |
  +--org.abora.gold.java.AboraHeaper
        |
        +--org.abora.gold.xpp.basic.Heaper
              |
              +--org.abora.gold.snarf.DiskManager
                    |
                    +--org.abora.gold.snarf.SnarfPacker

public class SnarfPacker

extends DiskManager

Should myFlocks contain full flockInfos for forwarded flocks? Both the flags and the size
mean nothing.
A SnarfPacker maintains the relationship between Shepherds and the set of snarfs
representing the disk. A SnarfPacker assigns flocks to snarfs based loosely on the
flocks''s Shepherd''s preferences. When a flock changes, it informs the SnarfPacker.
When the SnarfPacker decides to write to the disk, it ensures that the changed objects
still fit in their snarf (migrating them if necessary), writes them to the snarf, then
writes out the snarf.
mySnarfInfo {MuTable of: XuInteger}
- How much space remains in each snarf.
mySnarfMap {MuTable of: SnarfRecord}
- Map from snarfIDs to a SnarfRecord that handles that snarf.
myChangedSnarfs {MuSet of: XuInteger}
- The IDs for all snarfs in which an imaged flock has changed.
myFlocks {SetTable of: XuInteger and: FlockInfo}
- Indexed by Abraham hash, contains all FlockInfos that refer to flocks in memory.
Multiple infos may refer to the same flock if it is referenced through forwarding.
The only info considered to have the correct state wrt its flocks suitability for
purging is the info pointed to by its Abraham.
myInsideCommit {BooleanVar}
- True while writing new and changed flocks to disk to prevent purging,
and during purgeClean to prevent recursive call through Purgeror recycling.

Field Summary

protected static Collection	DebugSizes
protected static int	LRUCount
protected SnarfID	myAllocationSnarf
protected Cookbook	myBook
protected IntegerVar	myConsistentCount
protected IntegerVar	myDestroyCount
protected MuArray	myDestroyedFlocks
protected SetTable	myFlocks
protected boolean	myInsideCommit
protected IntegerVar	myLastNewCount
protected IntegerVar	myNewEstimate
protected IntegerTable	myNewFlocks
protected Counter	myNextHash
protected SanitationEngineer	myPurgeror
protected LiberalPurgeror	myRepairer
protected SnarfInfoHandler	mySnarfInfo
protected MuTable	mySnarfMap
protected Turtle	myTurtle
protected Urdi	myUrdi
protected UrdiView	myUrdiView
protected XcvrMaker	myXcvrMaker

Fields inherited from class org.abora.gold.snarf.DiskManager

myFlockInfoTable, myFlockTable, myFluidSpace, SecretEmulsion

Fields inherited from class org.abora.gold.xpp.basic.Heaper

AllBlasts, BecomeMap, GarbageCount, InGC, InitializedClasses, InitializingClasses, LastMemory, NextClientRequestNumber, NotOneElementSignal, PackageTable, PromiseNameTable, StringHashSBoxes

Fields inherited from class org.abora.gold.java.AboraHeaper

ActiveClubs, CurrentAuthor, CurrentBertCanopyCache, CurrentBertCrum, CurrentChunk, CurrentGrandMap, CurrentKeyMaster, CurrentPacker, CurrentSensorCanopyCache, CurrentServer, CurrentSession, CurrentSessions, CurrentTrace, InitialEditClub, InitialOwner, InitialReadClub, InitialSponsor, InsideTransactionFlag

Constructor Summary

SnarfPacker(Urdi urdi)

Method Summary

void	addInfo(FlockInfo info, Abraham shep) Add another flockInfo object to myFlocks with info about another location for shep.
void	assignSnarf(Abraham shep) Find a snarf in which to fit shep.
void	beginConsistent(IntegerVar dirtyFlocks) This is called before entering consistent block.
static void	bomb(SnarfPacker CHARGE)
void	checkInfos() Perform the sanity check of the moment.
void	commitState(boolean flag) Used by ResetCommit bomb
void	commitView() Commit by destroying the current view and creating a new one.
int	computeSize(Abraham flock) Send the snarf over a transmitter into a stream that just counts the bytes put into it.
void	consistentCount()
void	consistentX(IntegerVar dirty, BlockClosure aBlock)
UrdiView	currentView() Return the current urdiView.
void	destroyAbandoned() Destroy all forgotten flocks that are no longer in memory.
void	destroyFlock(FlockInfo info) Queue destroy of the given flock.
void	destruct() Destroy all objects imaged from this snarf.
void	diskUpdate(FlockInfo info) The flock described by info is Dirty!! On the next commit, rewrite it to the disk.
void	dismantleFlock(FlockInfo info) Turn the flock designated by info into a Pumpkin.
void	dropFlock(int token) The flock is being removed from memory.
void	endConsistent(IntegerVar dirty) This is called after exiting a consistent block.
Abraham	fetchCanonical(int hash, SnarfID snarfID, int index) If something is already imaged at that location, then return it.
Abraham	fetchInMemory(SnarfID snarfID, int index) Return true if the object is on disk but not in memory.
void	flushFlocks() Actually write all the changed and newly assigned flocks to the disk.
void	forgetFlock(FlockInfo info) Remember that there are no more persistent pointers to the shepherd represented by info.
MuSet	forgottenFlocks(SnarfID snarfID) Return the set of indices to flocks in snarf snarfID that are forgotten.
void	forwardFlock(Abraham shep) Shep has grown too large for its current place.
Abraham	getFlock(SnarfID snarfID, int index) Return a flock at a particular location.
Turtle	getInitialFlock() Return the starting object for the entire backend.
SnarfHandler	getReadHandler(SnarfID snarfID) Get the read handler on the snarf.
SnarfRecord	getSnarfRecord(SnarfID snarfID) Return the snarfRecord for snarfID.
static DiskManager	initializeUrdiOnDisk(java.lang.String fname)
boolean	insideCommit()
boolean	isFake()
static void	linkTimeNonInherited()
static Heaper	make(java.lang.String fname)
void	makePersistent() The virtual image in memory is now in a consistent state.
SpecialistRcvr	makeRcvr(XnReadStream readStream)
void	makeReal(FlockInfo info) Retrieve from the disk the flock at index within the specified snarf.
SpecialistXmtr	makeXmtr(XnWriteStream writeStream)
int	nextHashForEqual() Shepherds use a sequence number for their hash.
void	purge() Flush everything out to disk and remove all purgeable imaged objects from memory.
void	purgeClean()
void	purgeClean(boolean noneLocked) purge all shepherds that are currently clean, not locked, not dirty, and purgeable.
void	recordUpdate(FlockInfo info) The flock represented by info has changed.
void	refitFlocks() Make sure all flocks that have changed still fit in their snarfs. Add any that don't to myNewFlocks and return the table from their current locations to the newShepherds.
void	registerStub(Abraham shep, SnarfID snarfID, int index) Called to register a newly created stub (by the diskSpecialist) in the internal tables.
void	releaseReadHandler(SnarfHandler handler) Release the supplied snarfHandler and destroy it.
void	rememberFlock(FlockInfo info) There are now persistent pointers to the shepherd help by info.
FlockInfo	resolveLocation(FlockInfo info) Make sure that the shepherd or stub at that location actually points at the real location for a shepherd.
void	setHashCounter(Counter aCounter)
void	storeAlmostNewShepherd(Abraham shep) Do nothing
void	storeInitialFlock(Abraham turtle, XcvrMaker protocol, Cookbook cookbook) A turtle just got created!! Write out a pseudo-forwarder that has all the protocol information encoded in the snarfID and index.
void	storeNewFlock(Abraham shep) Shep just got created!! On some later commit, assign it to a snarf and write it to the disk.
void	testNewFlocks()

Methods inherited from class org.abora.gold.snarf.DiskManager

actualHashForEqual, bomb, cleanupGarbage, consistent, consistent, consistent, consistentBlockAt, emulsion, exitTimeNonInherited, flockInfoTable, flockInfoTable, flockTable, flockTable, fluidSpace, fluidSpace, initializeDisk, insistent, insistent, insistent, makeConsistent, makeConsistentBegin, makeConsistentEnd, staticTimeNonInherited

Methods inherited from class org.abora.gold.xpp.basic.Heaper

abstractDeclarationFor, abstractTypeFor, addMethodAttributeToInOf, addPackage, addPackageCategory, allClientProtocolOn, argumentTypesFor, arrow, blast, blast, BLAST, cachePromiseNameTable, cachePromiseNameTableIn, canYouBecome, cast, cleanPromiseClasses, clientClassesDo, clientFunctionsOn, clientMethodsOn, clientProtocol, clientProtocolDo, clientProtocolOn, clientProtocolOn, collectibleClasses, compare, compileClientSubclasses, compileConstantPromiseMethods, compileCreateFromRcvr, compileEQ, compileGeneratedClassMethod, compileGeneratedMethod, compileHook, compilePromise, compilePromiseDefaultMethods, compilePromiseFluidDeclarations, compilePromiseHandlers, compilePromiseMethods, compileRequestCreateMsgInArguments, compileRequestEvaluateMsgInReturningArguments, compileRPCSpecialistEvaluateMsgForReturningArguments, compileSendSelfTo, compileSendSelfToSendHook, compileStubbleMethods, compileSubclassStubbleMethods, computeMangle, computePreorder, constantTypeValue, convert, convertCopyDeclarations, convertDeferredDeclarations, convertProxyDeclarations, convertSubclassCopyDeclarations, convertSubclassDeferredDeclarations, convertSubclassProxyDeclarations, copyReferencesToType, create, create, create, create, create, create, create, create, create, create, createRequestClassArguments, definesProxyMethods, delete, deref, destroy, destructor, enum, enumFlags, equals, exportName, fetchAttribute, fetchPackage, fetchSuperCategory, fileOutClientProtocol, findCategory, findSenderAndReceiverMethods, findTailInto, flushPromiseNameTable, foo, freezeClientClasses, freezeClientProtocol, freezeStProtocol, frozenClasses, garbageCollect, garbageCollectFrom, gcOpportunity, gcOpportunity, generatedCategory, generatePromiseNames, getCategory, getOrMakePackage, getSuperCategory, handlerSignaturesFrom, hash, hashForEqual, hasProxyMethods, info_clientClasses, info_clientSideClasses, info_promiseClasses, info_stProtocol, inGC, initializedClasses, initializingClasses, initPackages, initStringHashSBoxes, inspectPieces, instanceSize, IntegerVar, isByProxy, isConstructed, isDestructed, isEqual, isEqualOrSubclassOf, isGenerated, isIntType, isKindOf, isRawType, isUnlocked, make, makeClassTable, makeFillTable, makeRequestTable, mangle, markChildren, markCount, markInstances, mayBecome, mayBecomeAnySubclassOf, new1, newX, nonCopyVariables, notWorking, pack, packageClasses, packagingCategory, parseExportName, passe, pointerToStaticMember, pointerToStaticMember, pointerToVirtualMember, preorderMax, preorderNumber, printOn, PROBLEM, problems, promiseClass, promiseDefaultValue, promiseName, promiseNameTable, promiseToAbstract, registerPackageCategory, removeGeneratedCode, removeStubbleMethods, removeSubclassGeneratedCode, removeSubclassStubbleMethods, requestProcedure, requestProceduresFrom, returnTypeFor, rootName, scheduleTermination, sendProxyTo, sendSelfTo, serverNameFor, setGC, signal, signals, smalltalkSelector, stClientProtocol, stubbleSelectorTokenReturnsArguments, subclassNonCopyVariables, takeOop, togglePromiseName, togglePromiseOfParse, unimplemented, unmangle, verifyFreeze, wipeStubble

Methods inherited from class org.abora.gold.java.AboraHeaper

asOop, basicInspect, displayString, error, hack, halt, inspect, knownBug, mightNotImplement, REQUIRES, shouldImplement, shouldNotImplement, stubbleForSubclassResponsibility, thingToDo, willNotImplement

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

mySnarfInfo

protected SnarfInfoHandler mySnarfInfo

myTurtle

protected Turtle myTurtle

myAllocationSnarf

protected SnarfID myAllocationSnarf

mySnarfMap

protected MuTable mySnarfMap

myFlocks

protected SetTable myFlocks

myNewFlocks

protected IntegerTable myNewFlocks

myLastNewCount

protected IntegerVar myLastNewCount

myNewEstimate

protected IntegerVar myNewEstimate

myDestroyedFlocks

protected MuArray myDestroyedFlocks

myUrdiView

protected UrdiView myUrdiView

myUrdi

protected Urdi myUrdi

myXcvrMaker

protected XcvrMaker myXcvrMaker

myBook

protected Cookbook myBook

myNextHash

protected Counter myNextHash

myConsistentCount

protected IntegerVar myConsistentCount

myInsideCommit

protected boolean myInsideCommit

myDestroyCount

protected IntegerVar myDestroyCount

myPurgeror

protected SanitationEngineer myPurgeror

myRepairer

protected LiberalPurgeror myRepairer

DebugSizes

protected static Collection DebugSizes

LRUCount

protected static int LRUCount

Constructor Detail

SnarfPacker

public SnarfPacker(Urdi urdi)

Method Detail

destroyFlock

public void destroyFlock(FlockInfo info)

Queue destroy of the given flock. The destroy will happen later.

Overrides:

destroyFlock in class DiskManager

diskUpdate

public void diskUpdate(FlockInfo info)

Description copied from class: DiskManager

The flock described by info is Dirty!! On the next commit, rewrite it to the disk.

Overrides:

diskUpdate in class DiskManager

dismantleFlock

public void dismantleFlock(FlockInfo info)

Turn the flock designated by info into a Pumpkin. It should have completed all dismantle
actions.

Overrides:

dismantleFlock in class DiskManager

dropFlock

public void dropFlock(int token)

The flock is being removed from memory. For now, this is an error
if the flock has been updated. If the flock has been forgotten,
then it will be dismantled when next it comes in from disk.
Because of forwarding, there may be many FlockInfos refering
to the flock if it is not new.

Overrides:

dropFlock in class DiskManager

forgetFlock

public void forgetFlock(FlockInfo info)

Remember that there are no more persistent pointers to the shepherd
represented by info. If it gets manually deleted, dismantle it immediately.
If it gets garbage collected, remember to dismantle it when it comes back
in from the disk.

Overrides:

forgetFlock in class DiskManager

getInitialFlock

public Turtle getInitialFlock()

Return the starting object for the entire backend. This will be the 0th
flock in the first snarf following the snarfInfo tables.

Overrides:

getInitialFlock in class DiskManager

nextHashForEqual

public int nextHashForEqual()

Shepherds use a sequence number for their hash. Return the next one
and increment. This should actually spread the hashes.

Overrides:

nextHashForEqual in class DiskManager

rememberFlock

public void rememberFlock(FlockInfo info)

There are now persistent pointers to the shepherd help by info.

Overrides:

rememberFlock in class DiskManager

storeAlmostNewShepherd

public void storeAlmostNewShepherd(Abraham shep)

Do nothing

Overrides:

storeAlmostNewShepherd in class DiskManager

storeInitialFlock

public void storeInitialFlock(Abraham turtle,
                              XcvrMaker protocol,
                              Cookbook cookbook)

A turtle just got created!! Write out a pseudo-forwarder that has all the protocol
information encoded in the snarfID and index.

Overrides:

storeInitialFlock in class DiskManager

storeNewFlock

public void storeNewFlock(Abraham shep)

Shep just got created!! On some later commit, assign it to a snarf
and write it to the disk.

Overrides:

storeNewFlock in class DiskManager

fetchCanonical

public Abraham fetchCanonical(int hash,
                              SnarfID snarfID,
                              int index)

If something is already imaged at that location, then return it. If there is already
an existing stub with the same hash at a different location, follow them till we
know that they are actually different objects.

Overrides:

fetchCanonical in class DiskManager

makeReal

public void makeReal(FlockInfo info)

Retrieve from the disk the flock at index within the specified snarf. Since
stubs are canonical, and this only gets called by stubs, the existing stub will
*become* the shepherd for the flock.

Overrides:

makeReal in class DiskManager

registerStub

public void registerStub(Abraham shep,
                         SnarfID snarfID,
                         int index)

Description copied from class: DiskManager

Called to register a newly created stub (by the diskSpecialist) in the internal
tables. The diskSpecialist in particular calls this when it couldn't find an
already existing stub (with fetchCacnonical) representing the flock at the
particular location.

Overrides:

registerStub in class DiskManager

addInfo

public void addInfo(FlockInfo info,
                    Abraham shep)

Add another flockInfo object to myFlocks with info about another location for shep.

computeSize

public int computeSize(Abraham flock)

Send the snarf over a transmitter into a stream that just counts the bytes put
into it.

currentView

public UrdiView currentView()

Return the current urdiView.

destroyAbandoned

public void destroyAbandoned()

Destroy all forgotten flocks that are no longer in memory.

forwardFlock

public void forwardFlock(Abraham shep)

Shep has grown too large for its current place. Treat it as just a new flock and give it
another place.

makeRcvr

public SpecialistRcvr makeRcvr(XnReadStream readStream)

makeXmtr

public SpecialistXmtr makeXmtr(XnWriteStream writeStream)

setHashCounter

public void setHashCounter(Counter aCounter)

Overrides:

setHashCounter in class DiskManager

testNewFlocks

public void testNewFlocks()

beginConsistent

public void beginConsistent(IntegerVar dirtyFlocks)

Description copied from class: DiskManager

This is called before entering consistent block. 'dirty' is the block's declaration of
the maximum number of shepherds which it can dirty. If this is a top level consistent
block, the virtual image in memory is now in a consistent state. It may be written to the
disk if necessary.

Overrides:

beginConsistent in class DiskManager

endConsistent

public void endConsistent(IntegerVar dirty)

Description copied from class: DiskManager

This is called after exiting a consistent block.

Overrides:

endConsistent in class DiskManager

insideCommit

public boolean insideCommit()

Overrides:

insideCommit in class DiskManager

makePersistent

public void makePersistent()

The virtual image in memory is now in a consistent state. Write the image of
all changed or new Shepherds out to the disk in a single atomic action. The
atomicity only happens on top of a real Urdi, however.

purge

public void purge()

Flush everything out to disk and remove all purgeable imaged
objects from memory.

Overrides:

purge in class DiskManager

purgeClean

public void purgeClean(boolean noneLocked)

purge all shepherds that are currently clean, not locked, not dirty,
and purgeable. Purging just turns them into stubs, freeing all their
flocks. Garbage collection can clean up the flocks and any stubs no
longer pointed to by something in memory. Because infos for new
flocks don't appear in myFlocks, this will not throw out any newFlocks
(which will be marked dirty anyway). For each FlockInfo, we check
that its flock refers to that exact instance to get correct information
about its dirty state.

Overrides:

purgeClean in class DiskManager

destruct

public void destruct()

Destroy all objects imaged from this snarf.

Overrides:

destruct in class DiskManager

assignSnarf

public void assignSnarf(Abraham shep)

Find a snarf in which to fit shep. Then assign it to
that location, and mark that snarf as changed.

checkInfos

public void checkInfos()

Perform the sanity check of the moment. Beware the compile cost of changing this comment.

commitState

public void commitState(boolean flag)

Used by ResetCommit bomb

commitView

public void commitView()

Commit by destroying the current view and creating a new one.

fetchInMemory

public Abraham fetchInMemory(SnarfID snarfID,
                             int index)

Return true if the object is on disk but not in memory.

flushFlocks

public void flushFlocks()

Actually write all the changed and newly assigned flocks to the disk.

forgottenFlocks

public MuSet forgottenFlocks(SnarfID snarfID)

Return the set of indices to flocks in snarf snarfID that are forgotten.

getFlock

public Abraham getFlock(SnarfID snarfID,
                        int index)

Return a flock at a particular location. This needs to register
the flock if it doesn't exist already.

getReadHandler

public SnarfHandler getReadHandler(SnarfID snarfID)

Get the read handler on the snarf.

getSnarfRecord

public SnarfRecord getSnarfRecord(SnarfID snarfID)

Return the snarfRecord for snarfID. The SnarfRecord must exist if there are
changed flocks imaged out of that snarf, but might not otherwise. Create it if necessary.

recordUpdate

public void recordUpdate(FlockInfo info)

The flock represented by info has changed. Record it in the
bookkeeping data-structures. This must be called by all things
that affect whether the flock gets rewritten to disk.

refitFlocks

public void refitFlocks()

Make sure all flocks that have changed still fit in their snarfs.
Add any that don't to myNewFlocks and return the table
from their current locations to the newShepherds.

releaseReadHandler

public void releaseReadHandler(SnarfHandler handler)

Release the supplied snarfHandler and destroy it.

resolveLocation

public FlockInfo resolveLocation(FlockInfo info)

Make sure that the shepherd or stub at that location actually points
at the real location for a shepherd. This will resolve forwarding pointers,
but not instantiate any flocks.

consistentCount

public void consistentCount()

purgeClean

public void purgeClean()

consistentX

public void consistentX(IntegerVar dirty,
                        BlockClosure aBlock)

isFake

public boolean isFake()

Overrides:

isFake in class DiskManager

linkTimeNonInherited

public static void linkTimeNonInherited()

initializeUrdiOnDisk

public static DiskManager initializeUrdiOnDisk(java.lang.String fname)

make

public static Heaper make(java.lang.String fname)

bomb

public static void bomb(SnarfPacker CHARGE)