Housekeeping

Intervall and Time Budget

Housekeeping interval and time budget is configured by setting up a StorageHousekeepingController.

Available properties are:

Property Description

housekeeping-interval

Interval the housekeeping is triggered in milliseconds, default once per every second

housekeeping-time-budget

Time budget for housekeeping in nanoseconds, default is 0.01 seconds

housekeeping-adaptive

Usage of an adaptive housekeeping controller, which will increase the time budgets on demand, if the garbage collector needs more time to reach the sweeping phase.

housekeeping-increase-threshold

The threshold of the adaption cycle to calculate new budgets for the housekeeping process. Default is 5 seconds.

housekeeping-increase-amount

The amount the housekeeping budgets will be increased each cycle. Default is 50 ms.

housekeeping-maximum-time-budget

The upper limit of the housekeeping time budgets. Default is 0.5 seconds.

Enabling Adaptive Housekeeping for Write-Intensive Data Applications

EclipseStore is designed to write data as soon as it is requested. If there is a high volume of consecutive data writes, the housekeeping process does not start, resulting in an increased number of open files and significant disk space consumption. One possible solution is to use the Adaptive Housekeeping Controller, which, based on its configuration, allocates time for cleaning and reorganizing data files on the disk.

// New foundation
final EmbeddedStorageFoundation<?> foundation = EmbeddedStorage.Foundation();

// Housekeeping controller with fixed time budget
final StorageHousekeepingController fixedController = Storage.HousekeepingController(
	1000, // interval in ms
	10_000_000 // time budget in ns
);

// Wrap in adaptive controller
final StorageHousekeepingController adaptiveController = StorageHousekeepingController
	.AdaptiveBuilder(fixedController)
	.increaseThresholdMs(100)
	.increaseAmountNs(Duration.ofMillis(1).toNanos())
	.maximumTimeBudgetNs(Duration.ofMillis(100).toNanos())
	.buildFor(foundation)
;

// Use controller in foundation
foundation.setConfiguration(Storage.ConfigurationBuilder()
	.setHousekeepingController(adaptiveController)
	.createConfiguration()
);

// Start storage
final EmbeddedStorageManager storage = foundation.start();

File Sizes and Payload

The desired file min and max sizes and payload ratio is configured by the StorageDataFileEvaluator:

available properties are:

Property Description

data-file-minimum-size

Files smaller then minimum file size will be merged with other files if possible, default is 1 MB.

data-file-maximum-size

Files larger then maximum file size will be split in smaller ones, default is 8 MB.

data-file-minimum-use-ratio

Ratio of non-gap data contained in a storage file to prevent the file from being dissolved, default is 0.75 (75%).

 
EmbeddedStorageManager storage = EmbeddedStorage.Foundation(
	Storage.ConfigurationBuilder()
		.setDataFileEvaluator(Storage.DataFileEvaluator(1024*1024, 1024*1024*8, 0.75))
		.createConfiguration())
	.start();

Cache

The lifetime of objects in the internal entity cache can be configured by the StorageEntityCacheEvaluator:

Available properties are:

Property Description

entity-cache-threshold

Abstract threshold value, roughly comparable to size in bytes with a time component, at which a cache must be cleared of some entities. Default is 1000000000.

entity-cache-timeout

Time in milliseconds after that an entity is considered to be old if not read meanwhile. Must be greater zero, default is 86400000ms (1 day).

 
EmbeddedStorageManager storage = EmbeddedStorage.Foundation(
	Storage.ConfigurationBuilder()
		.setEntityCacheEvaluator(Storage.EntityCacheEvaluator(
			86_400_000,
			1_000_000_000))
		.createConfiguration())
	.start();
For external configuration see: Properties