Skip to content

Feature: support event extraction on the ABF format - #1961

Open
h-mayorquin wants to merge 2 commits into
NeuralEnsemble:masterfrom
h-mayorquin:axon_tag_events
Open

h-mayorquin wants to merge 2 commits into
NeuralEnsemble:masterfrom
h-mayorquin:axon_tag_events

Conversation

@h-mayorquin

Copy link
Copy Markdown
Contributor

Tags are the markers an experimenter adds to an Axon file during a recording, like a comment typed when a drug is washed on or a pulse from an external trigger. I checked how AxonRawIO reads them against pyABF and against Clampex itself, and right now only version 2 gap-free files with one channel come out right. The current problems are:

  • The times are 4 times too large in version 2 episodic files, 2 times too large in gap-free files with two channels and garbage in every version 1 file.
  • The label is the tag type code ('1') instead of the comment.
  • The comments annotation is the b'...' repr of the bytes.
  • Every segment gets every tag and the segments of episodic files stretch to cover all of them.
  • All the tag types go to a single Tag channel with an empty id, even in files without tags.

This PR fixes all of this. lTagTime counts in fSynchTimeUnit microseconds, or in samples of the interleaved data when fSynchTimeUnit is 0 (this is what ABFHEADR.H and the vendor library converter ABFH_SynchCountToMS say), and the t_start and t_stop window now uses the same conversion. For version 1 the reader multiplies lTagSectionPtr by the 512-byte block size before seeking. The comment is now the label, and each tag type present in the file gets its own event channel with the name and id from the header constants (time_tag, comment_tag, external_tag, voice_tag, new_file_tag, annotation_tag). Each segment gets only the tags inside its sweep, and a tag that falls between sweeps goes to no segment. I checked the times against pyABF on the 60 tagged files I have and they agree except for a version 1 file with 4 channels, where pyABF scales by fADCSampleInterval / nADCNumChannels and gets half the time the header gives.

This changes what users get: a file without tags has no event channel instead of an empty Tag channel, event channel 0 is the first tag type in the file, and the comments annotation is gone as the comment is the label now. I preferred to break this once instead of in several steps, and since the reader was right for one kind of file only there is little working behavior to keep. The test files were added to gin in https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/pulls/197. I also read the tag section with one np.frombuffer call instead of a loop over the entries. The new code picks the tags of each type and each sweep with array masks and this needs each field as an array, not a list of dicts.

Convert tag times with fSynchTimeUnit (or as samples of the interleaved data when it is 0), seek the ABF1 tag section in 512-byte blocks, use the comment as the label, give each tag type its own event channel and give each segment only the tags inside its sweep.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant