Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tempfiles #80

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
ae11869
get_valid_path now can look for presence of a file
mmagnuski Jun 20, 2015
9460739
add docstring to get_valid_path
mmagnuski Jun 20, 2015
5162dc4
separate out db_epoch from recoverEEG
mmagnuski Jun 20, 2015
be9f20b
haspostfilter changes the moment when epoching is performed
mmagnuski Jun 20, 2015
063e67c
adds tempsave behavior to recoverEEG
mmagnuski Jun 20, 2015
34b44c8
move ends so that versions are not updated when compsgui has just opened
mmagnuski Jun 20, 2015
2e6beab
adapt db_gui to use tempsave
mmagnuski Jun 20, 2015
f3c5d30
recoverEEG cleanup
mmagnuski Jun 20, 2015
03c7ac4
this may be completely unnecessary - make sure navigation uses fresh …
mmagnuski Jun 20, 2015
a4a74b3
change onesecepoch to take EEG as the first arg and remove filepath, …
mmagnuski Jul 8, 2015
2d46bc5
move recovery in gui to separate function
mmagnuski Jul 8, 2015
535ddf1
fixes db_gui_recover refreshing
mmagnuski Jul 9, 2015
d8c5b24
get_valid_path now accepts 'dir' option and looks for a path containi…
mmagnuski Jul 9, 2015
0f32520
Merge branch 'ft_tempfiles' of https://github.com/mmagnuski/eegDb int…
mmagnuski Jul 9, 2015
fcb66b0
modify db_temp_get to work with '+dir'
mmagnuski Jul 9, 2015
076aebb
fix typo in db_temp_get
mmagnuski Jul 9, 2015
cb87063
move checking tempfile path to separate function
mmagnuski Jul 9, 2015
85753ce
init pth to empty in get_valid_path
mmagnuski Jul 9, 2015
0397b19
db_temp_get also returns ind now
mmagnuski Jul 9, 2015
adde545
fist version of tempfile clearing
mmagnuski Jul 9, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions db/db_clear_temp.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
function db = db_clear_temp(db, rs)

% DB_CLEAR_TEMP allows to clear unused temporary files from disc
% (or remove from the database information about temporary files
% that no longer exist)

if ~exist('rs', 'var')
rs = 1:length(db);
end
% remove unused temp files
for r = rs
[tmp, mtch, ind] = db_temp_get(db, r);
if ~mtch
db = db_remove_temp(db, r, 'all');
else
temp_inds = 1:length(db(r).datainfo.tempfiles);
temp_inds(ind) = [];
db = db_remove_temp(db, r, temp_inds);
end
end
115 changes: 115 additions & 0 deletions db/db_epoch.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
function EEG = db_epoch(EEG, db, r, segmentinfo)

% DB_EPOCH is the function used by eegDb for epoching
% files are epoched automatically when they are recovered
% FIXHELPINFO


if femp(db(r), 'epoch')

% defaults
% --------
code_id = '\code:';
cidlen = length(code_id);

if ~exist('r', 'var')
r = 1;
end

if ~exist('segmentinfo', 'var')
% is segment given:
if isfield(db(r).epoch, 'segment') && ...
~isempty(db(r).epoch.segment) && isnumeric(db(r).epoch.segment)
segmentinfo = true;
end
end

% windowing
% ---------
% (consecutive epochs, not event locked)
if femp(db(r).epoch, 'locked') && ~db(r).epoch.locked

% ==============
% onesec options
options.fill = true;

flds = {'filter', 'winlen', 'distance',...
'leave', 'eventname'};

% checking fields for onesecepoch
for f = 1:length(flds)
if femp(db(r).epoch, flds{f})
options.(flds{f}) = db(r).epoch.(flds{f});
end
end

% if prerej is present then no need to use distance
if femp(db(r).reject, 'pre')
options.distance = [];
end

% ===================
% call to onesecepoch
EEG = onesecepoch(EEG, options);
clear options

elseif ~isempty(db(r).epoch.events) && ...
~isempty(db(r).epoch.limits)

% ==================
% classical epoching
epoc = db(r).epoch.events;

% checking for code generator of epochs
% ADD - function handle for epoching?
% or maybe not necessary - there is an
% option for user-defined function
if ischar(epoc) && length(epoc) > cidlen && ...
strcmp(epoc(1:cidlen), code_id)

epoc = eval(epoc(cidlen+1:end));
end

EEG = db_fastepoch(EEG, epoc, db(r).epoch.limits);

% =======================
% checking for segmenting
if segmentinfo && ~nosegment
EEG = segmentEEG(EEG, db(r).epoch.segment);
end
end
end

% CHANGE
% [ ] if we segment then orig_numep should be adjusted too
% [ ] instead of numep this all can be done in a smarter way
% 1) generally - onesec can add numep too
% 2) numep can be inferred from length of rejections
% in db(r)!
% 3) ...
% [ ] in the current version only onesecepoching is checked
% for while in future releases we want to include also
% conditional epoch extraction (only correct etc.) which
% is another prerej
%
%
% ======================
% adding orig_numep info
%
% (this is later used when rejections are added
% using a recovered file that has some of the
% rejections already removed)
%
% if epoched signal add orig_numep
EEG.etc.orig_numep = size(EEG.data, 3);

% if onesecepoch was perfromed add onesec info
if femp(db(r).epoch, 'locked') && ~db(r).epoch.locked

% either prerej is nonempty % or what?
if femp(db(r).reject, 'pre')
% there is some info about prerej,
% we correct orig_numep
EEG.etc.orig_numep = EEG.etc.orig_numep - length(db(r).reject.pre);
end
end
20 changes: 20 additions & 0 deletions db/paths/db_remove_temp.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
function db_remove_temp(db, r, optind)

%
ind = optind;
if ischar(optind)
if strcmp(optind, 'all')
ind = 1:length(db(r).datainfo.tempfiles);
end
end
ind = ind(:)';

% remove file
for rmv = ind
pth = db_test_temp_path(db, r, rmv);
delete(fullfile(pth, db(r).datainfo...
.tempfiles(rmv).filename));
end

% clear info from the db
db(r).datainfo.tempfiles(ind) = [];
106 changes: 106 additions & 0 deletions db/paths/db_temp_get.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
function [tempdb, match, ind] = db_temp_get(db, r)

% DB_TEMP_GET gives info on relevant tempfile for db
% record r.
%
% usage
% -----
% temp = db_temp_get(db, r);
%
% input
% -----
% db - *structure*; eegDb database
% r - *integer*; record number
%
% output
% ------
% temp - *structure*; tempfile info, with fields:
% .filename
% .filepath
% .filter
% .cleanline
% .epoch



tempdb = [];
match = 0;

if femp(db(r).datainfo, 'tempfiles')
% some tempfiles are present, check which
% ones are present on disc
temp = db(r).datainfo.tempfiles;
tempok = false(1, length(temp));
for t = 1:length(temp)
tempok(t) = ~isempty(db_test_temp_path(db, r, t));
end
temp = temp(tempok);

if isempty(temp)
return
end

% check which tempfile matches the
% current database pipeline

% get filtering and epoching from the current record:
tst = struct('filter', [], 'epoch', [], 'cleanline', []);

% epoching
tst.epoch = db_getepoching(db(r));

% filtering (should CHANGE to db_get later on)
in = cellfun(@(x) femp(x, 'filter'), {db(r), db(r).datainfo});
in(3) = femp(db(r).datainfo, 'filtered');

if any(in)
if in(1)
tst.filter = db(r).filter;
elseif in(2)
tst.filter = db(r).datainfo.filter;
elseif in(3)
tst.filter = db(r).datainfo.filtered;
end
end

% cleanline
if femp(db(r).datainfo, 'cleanline')
tst.cleanline = db(r).datainfo.cleanline;
end

% only filtering and epoching are considered
checkflds = {'cleanline', 'filter', 'epoch'};
test_temp = false(length(checkflds), length(temp));

for f = 1:length(checkflds)
test_temp(f, :) = cellfun(@(x) isequal(x, ...
tst.(checkflds{f})), {temp.(checkflds{f})});
end
good_temp = all(test_temp(1:2, :), 1);
best_temp = all(test_temp, 1);

ind = [];
% choose first one matching (CHANGE!)
if any(best_temp)
ind = find(best_temp, 1);
match = 2;
elseif any(good_temp)
ind = find(good_temp, 1);
match = 1;
end

if ~isempty(ind)
% give tempdb
tempdb = struct();
tempdb.filename = temp(ind).filename;
if temp(ind).filepath(1) == '+'
tempdb.filepath = get_valid_path(temp(ind).filepath, ...
'file', temp(ind).filename, 'dir', temp(ind).filepath(2:end));
else
tempdb.filepath = temp(ind).filepath;
end
tempdb.filter = temp(ind).filter;
tempdb.epoch = temp(ind).epoch;
tempdb.cleanline = temp(ind).cleanline;
end
end
15 changes: 15 additions & 0 deletions db/paths/db_test_temp_path.m
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
function pth = db_test_temp_path(db, r, t)

temp = db(r).datainfo.tempfiles;
if temp(t).filepath(1) == '+'
% this uses a vild path from db(r).filepath
% and a subdir in it
pth = get_valid_path(db(r).filepath, ...
'file', temp(t).filename, ...
'dir', temp(t).filepath(2:end), ...
'noerror', true));
else
pth = get_valid_path(temp(t).filepath, ...
'file', temp(t).filename, ...
'noerror', true));
end
80 changes: 70 additions & 10 deletions db/paths/get_valid_path.m
Original file line number Diff line number Diff line change
@@ -1,6 +1,51 @@
function pth = get_valid_path(PTH)
function pth = get_valid_path(PTH, varargin)

% FIXHELPINFO
% GET_VALID_PATH checks which path is valid on a given machine
%
% usage
% -----
% pth = get_valid_path(PTH, ...);
%
% input
% -----
% PTH - *string* or *cell array of strings*; path string(s)
%
% ... - additional key - value pairs
%
% key-value pairs
% ---------------
% 'file' - *string*; filename that should be present
% in the valid path
% 'dir' - *string*; directory that should be present
% in the valid path. In conjunction with 'file'
% option - the file has to be in the directory
% denoted by 'dir'
% 'noerror' - *logical*; whether NOT to throw an error when valid
% path was not found
% default: false (error is thrown)
%
% examples
% --------
% check which path to Dropbox is correct on given machine:
% PTH = {'D:\Dropbox\', 'C:\Users\Kant\Dropbox\'};
% validpath = get_valid_path(PTH);
%
% check which folder exists AND has file 'data.set' inside:
% PTH = {'C:\DATA\superstudy\', 'C:\DATA\otherstudy\'};
% validpath = get_valid_path(PTH, 'file', 'data.set');
%
% see also: isdir

% additional args
pth = [];
opt.file = [];
opt.dir = [];
opt.noerror = false;
if nargin > 1
opt = parse_arse(varargin, opt);
end
iffile = ~isempty(opt.file);
ifdir = ~isempty(opt.dir);

% if not cell - close in a cell
if ~iscell(PTH)
Expand All @@ -11,13 +56,28 @@
fnd = false;
for p = 1:length(PTH)
% and test isdir() on them
if isdir(PTH{p})
% if it is dir, stop looking
pth = PTH{p};
fnd = true;
break
thispth = PTH{p};
if ifdir
thispth = fullfile(thispth, opt.dir);
end
if isdir(thispth)
% if filecheck, check if file is present:
if iffile
fls = dir(thispth);
fileok = any(strcmp(opt.file, {fls(~[fls.isdir]).name}));
end

% if everything ok - stop looking
if ~iffile || fileok
pth = thispth;
fnd = true;
break
end
end
end

if ~opt.noerror
if ~fnd
error('Could not find the correct path');
end
end
if ~fnd
error('Could not find the correct path');
end
Loading