The CoreLayer

You can import from ./ or with bfly installed.

from bfly import CoreLayer

Or, in scripts in some directories:

# if './bfly/' in sys.path
import CoreLayer

You can from CoreLayer import All layers.

# if './bfly/' in sys.path
from CoreLayer import DatabaseLayer, AccessLayer
from CoreLayer import QueryLayer, ImageLayer, UtilityLayer

CoreLayer

You can import CoreLayer from bfly.

CoreLayer classes

class CoreLayer.Core(db)

Starts the Cache

Parameters:db (bfly.Butterfly._db_type) – A fully-loaded database

_db

bfly.Butterfly._db_type – Taken from first argument db

_cache

Cache – Able to store images and metadata using UtilityLayer.RUNTIME instance from db argument

find_tiles(d_query)

Load the requested image for a DataQuery

Parameters:

• d_query (DataQuery) – Request for a scaled subvolume of a source image
• Returns –
• numpy.ndarray – The full image data for the requested region

find_unique(d_query)

Get unique values for a DataQuery

Parameters:

• d_query (DataQuery) – Request for a scaled subvolume of a source image
• Returns –
• set – The set of unique values for the request

get_data(d_query)

dumps answer to d_query as a string

Calls update_query() with more informationfrom the cache or from the properties of a tile. Also calls find_tiles() to get the completeimage needed to answer the d_query.

Parameters:i_query (QueryLayer.InfoQuery) – A request for information Returns:Answer for the QueryLayer.InfoQuery Return type:str

get_dataset(i_query)

dumps dataset from i_query as a string

Calls update_query() with more informationfrom the cache or from the properties of a tile.

Parameters:i_query (QueryLayer.InfoQuery) – A request for information Returns:Dataset info for QueryLayer.InfoQuery Return type:str

get_edits(i_query, msg={})

dumps websocket updates to i_query as a string

Calls update_query() with more informationfrom the cache or from the properties of a tile.

Parameters:i_query (QueryLayer.InfoQuery) – A request for information Returns:Wesocket info for QueryLayer.InfoQuery Return type:str

static get_groups(i_query)

dumps group list for i_query as a string

Parameters:i_query (QueryLayer.InfoQuery) – A request for a list of groups Returns:A list of all groups for the i_query Return type:str

get_info(i_query)

dumps answer to i_query as a string

Calls update_query() with more informationfrom the cache or from the properties of a tile.

Parameters:i_query (QueryLayer.InfoQuery) – A request for information Returns:Channel info for QueryLayer.InfoQuery Return type:str

load_tile(t_query)

Load a single tile from the cache or from disk

Parameters:t_query (TileQuery) – With tile coordinates and volume within the tile Returns:The subregion image data for the requested tile Return type:numpy.ndarray

static make_data_query(i_query)

Make a data query from an info query

Parameters:i_query (InfoQuery) – only needs PATH set in OUTPUT.INFO Returns:takes only the PATH from i_query Return type:DataQuery

static make_tile_query(d_query, t_index=array([0, 0, 0], dtype=uint32))

Make a TileQuery from DataQuery

Parameters:

• d_query (DataQuery) – only needs PATH set in OUTPUT.INFO
• t_index (numpy.ndarray) – The 3x1 count of tiles form the origin

Returns:

One tile request in the given data request

Return type:

TileQuery

update_query(query)

Finds missing query details from cache or tile

Makes keywords from either the _cache or from a new TileQuery to update the given query

Parameters:query (Query) – Either an InfoQuery or a DataQuery Returns:keywords – Can pass to Query.update_source() or combine to pass to Query.update_dataset(). Return type:dict

static view_volume(view, vol)

Display a volume in color or grayscale

Parameters:

• view (str) – The requested color or gray view of the data
• vol (str) – Raw volume from Cache / Datasource

Returns:

Colorized or original raw volume

Return type:

numpy.ndarray

write_image(d_query, volume)

Format a volume for a given DataQuery

Parameters:

• d_query (DataQuery) – With the format and view for the requested volume
• volume (numpy.ndarray) – Raw volume from Cache / Datasource

Returns:

The image response as a formatted bytestring

Return type:

str

class CoreLayer.Cache(_runtime)

Cache tiles and preloaded keywords

Parameters:_runtime (RUNTIME) – Needed to make attributes for each instance

_max_memory

int – MAX from RUNTIME.CACHE, and the max bytes of memory to be used.

_cach_meta

str – MAX from RUNTIME.CACHE, and the key for the size of cached keywords

_cache

Collections.OrderedDict – A Least recently used ordered dictionary

_now_memory

int – The current bytes of memory used

get(key)

Get a value from the cache by key

Parameters:key (str) – The key from a Query to access the cache Returns:The value stored in the cache, or an empty list Return type:anything

set(key, value)

Set a key in the cache to a value

Parameters:

• key (str) – The key from a Query to access the cache
• value (anything) – The keywords or tile value to store in the cache

Returns:

0 if successful and -1 if value is too large

Return type:

int

value_size(value)

Get actual memory size of a value

Parameters:value (anything) – expected to be a dict or a numpy.ndarray Returns:the number of bytes used by the value Return type:int

DatabaseLayer

You can import DatabaseLayer from bfly and CoreLayer.

DatabaseLayer classes

Database base class

class DatabaseLayer.Database(path, _runtime)

Stores tables to respond to API._feature_info()

Parameters:

• path (str) – The file path to store and access the database
• _runtime (RUNTIME) – Gets stored as RUNTIME

RUNTIME

RUNTIME – With keywords needed to load files and use tables

add_entries(table, path, t_keys, entries)

Add an array or list of entries to a table Must be overridden by derived class.

add_entry(table, path, entry, update=1)

and a single entry to a table for a path Overides Database.add_entry()

Parameters:

• table (str) – The category of table for the database
• path (str) – The dataset path to metadata files
• entry (dict) – The mapping of keys to values for the entry
• update (int) – 1 to update old entries matching keys, and 0 to write new entries ignoring old entries. Default 1.

Returns:

The value of the entry

Return type:

dict

add_neurons(path, neurons)

Add all the neurons to the database

Parameters:

• path (str) – The dataset path to metadata files
• neurons (numpy.ndarray) – The Nx4 array of id, z, y, x values where N is the number of neurons for the path.

Returns:

A list of dicts from each row of neurons with dictionary keys from the NEURON.FULL_LIST field of RUNTIME.DB

Return type:

list

add_path(c_path, d_path)

store a link from a c_path to a d_path Must be overridden by derived class.

Parameters:

• c_path (str) – The path to a given channel with images
• d_path (str) – The path to the dataset with metadata files

add_synapses(path, synapses)

Add all the synapases to the database

Parameters:

• path (str) – The dataset path to metadata files
• synapses (numpy.ndarray) – The Nx5 array of pre, post, z, y, x values where N is the number of synapses for the path.

Returns:

A list of dicts from each row of synapses with dictionary keys taken from SYNAPSE.FULL_LIST field of RUNTIME.DB

Return type:

list

add_table(table, path)

Add a table to the database Must be overridden by derived classes

Parameters:

• table (str) – The category of table for the database
• path (str) – The dataset path to metadata files

Returns:

The table name combining table and path The derived classes should return whether the table was added successfully.

Return type:

str or bool

add_tables(path)

Store all the tables for a given path

Parameters:path (str) – The dataset path to metadata files

all_neurons(table, path)

Must be overridden by derived class.

commit()

Save all database changes to the database file. Must be overridden by derived class.

get_by_key(table, path, key)

Get the entry for the key in the table. Must be overridden by derived class.

Parameters:

• table (str) – The category of table for the database
• path (str) – The dataset path to metadata files
• key (int) – The primary key value for any entry

Returns:

The object reference from get_table(). The derived class should give an entry in the table.

Return type:

object or dict

get_path(path)

Map a channel path to a dataset path Must be overridden by derived class.

Parameters:path (str) – The path to the given channel Returns:The dataset path for the given path Return type:str

get_table(table, path)

Get the actual table for a given path Must be overridden by derived class.

Parameters:

• table (str) – The category of table for the database
• path (str) – The dataset path to metadata files

Returns:

Full database name of the table for a path. The derived classes should actually return the python object reference to the real table.

Return type:

str or object

is_neuron(table, path, id_key)

Must be overridden by derived class.

is_synapse(table, path, id_key)

Must be overridden by derived class.

load_all(source)

Load the tables, synapses, and neuron configs :param source: The configuration options for a dataset :type source: dict

load_config(config)

Loads all files from config into database

Parameters:config (dict) – all data from UtilityLayer.rh_config Returns:the derived database class instance Return type:Database

load_neurons(path, synapses)

Load all the neuron information from files

Parameters:

• path (str) – The dataset path to metadata files
• synapses (numpy.ndarray) – The Nx5 array of pre, post, z, y, x values where N is the number of synapses for the path.

Returns:

The Nx4 array of id, z, y, x values where N is the number of neurons for the path.

Return type:

numpy.ndarray

load_synapses(path)

Load all the synapse information from files

Parameters:path (str) – The dataset path to metadata files Returns:The Nx5 array of pre, post, z, y, x values where N is the number of synapses for the path. Return type:numpy.ndarray

neuron_children(table, path, id_key, start, stop)

Must be overridden by derived class.

neuron_keypoint(table, path, id_key, scales)

Must be overridden by derived class.

synapse_ids(table, path, start, stop)

Must be overridden by derived class.

synapse_keypoint(table, path, id_key, scales)

Must be overridden by derived class.

synapse_parent(table, path, id_key)

Must be overridden by derived class.