-
Notifications
You must be signed in to change notification settings - Fork 7
/
Copy pathNEWS
2739 lines (2004 loc) · 117 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
New in version 0.11.0:
Library Changes:
* BUG FIX: When using a bzipped /REFERENCE field, mixing gd_nframes()
calls with reads of data from that field no longer results in
subsequent reads to occur in the wrong place. Reported by Matthew
Petroff.
* BUG FIX FOR CVE-2021-20204: The first RAW field in a Dirfile having
the same name as a previously defined field no longer results in a
segmentation fault when trying to access the Dirifle's reference
field.
By default, the parser will return a syntax error (GD_E_FORMAT)
when encountering a duplicate field name, making the DIRFILE
invalid and avoiding the segmentation fault. To trigger the
segmentation fault, the standard GD_E_FORMAT error return
must be avoided, by doing one of the following:
- The GD_IGNORE_DUPS flag must be used when invoking the format
file parser. This stops the parser from generating an error
when discarding out the duplicate field.
- A caller-supplied parser callback must return GD_SYNTAX_IGNORE
when passed the GD_E_FORMAT error generated by the duplicate
field. (This is what happens in the checkdirfile utility,
which is why it is affected by this bug.)
API Changes:
* A new function gd_open_limit() can be used to limit the number of
open file descriptors used by the library to access RAW data in
a given DIRFILE. GetData will try to keep under the limit by
automatically closing RAW fields when necessary. Note: the
library may exceed the limit in certain cases. See the man
page for full details.
* A number of symbols, which were deprecated in GetData-0.8,
have been removed. This removal extends to the corresponding
symbols in the bindings. The removed symbols, and their
replacements, are:
Removed symbol Replacement
------------------- -------------------
GD_FLOAT GD_FLOAT32
GD_DOUBLE GD_FLOAT64
GD_E_BAD_ENDIANNESS GD_E_ARGUMENT
GD_E_BAD_PROTECTION GD_E_ARGUMENT
GD_E_BAD_VERSION GD_E_ARGUMENT
GD_E_FORMAT_NO_PARENT GD_E_FORMAT_NO_FIELD
gd_bit_t int
gd_spf_t unsigned int
NOTE: The Python bindings have a different FLOAT type alias which
corresponds to the native float type in Python. That symbol is
not deprecated and has not been removed.
* BUG FIX: Lingering references to the type gd_shift_t, which was
deprecated in GetData-0.10.0 and replaced by gd_int64_t, have been
removed from the API.
Bindings Changes:
* PYTHON BUG FIX: When first_sample is non-zero, and neither num_frames
nor num_samples is indicated dirfile.getdata() no longer attempts to
read past the end of the data file. Patch from Matthew Petroff.
Miscellaneous
* Running "make clean" no longer accidentally deletes most of the man pages
from the man/ subdirectory.
* New options --with-pcre and --without-pcre have been added to ./configure
to adjust how the PCRE library is (or is not) used.
* Modules now link to the main getdata library to avoid the potential for
symbol look-up errors on load. Reported by Matthew Petroff.
* Fortran bindings can now be built with gfortran version >= 10.
|=========================================================================|
New in version 0.10.0:
Dirfile Changes:
* Dirfile Standards Version 10 has been released. It adds three new
field types: SARRAY, which is an array of STRING scalars (like a
CARRAY, but with STRINGs instead of CONSTs), and INDIR and SINDIR,
which are vector fields provide indexed look-ups from CARRAY (INDIR) or
SARRAY (SINDIR) scalar arrays. It also adds field code namespaces,
which can be specified with the new /NAMESPACE directive, or else as
part of an /INCLUDE directive.
* Some notes about namespaces:
- Namespaces are separated from field names by a dot (.):
namespace.name
and can be nested arbitrarily deep, separating namespaces with
intermediate dots:
namespace.subspace.subsubspace.name
Namespaces were created to provide an alternative to the prefix and
suffix added to the /INCLUDE directive in Standards Version 9, which
have some unfortunate side-effects due to their modifying field names
directly. Because namespaces nest and are syntactically separate
from field names, they do a better job of encapsulation.
- If the namespace of a field is null (""), which is the default, then
the dot separating the namespace from the field name may be omitted.
- An /INCLUDE statement can specify a namespace which becomes the
included fragment's "root namespace". The root namespace of the base
(top-level) format file is always the nullspace (""), and cannot be
changed.
- In addition to root namespaces, there is also a "current namespace".
At the top of a fragment, the current namespace is set to the root
namespace. A /NAMESPACE directive can be used to change the current
namespace to a subspace under the root namespace. That is, if the
root namespace is "root", then the directive:
/NAMESPACE subspace
changes the current namespace to "root.subspace" regardless of what
the current namespace was before. To change the current namespace
back to the root namespace, use the null token ("") with the
/NAMESPACE directive:
/NAMESPACE ""
- If no namespace is specified in an /INCLUDE line, then the current
namespace becomes the root namespace of the included fragment.
- Subnamespaces under the root namespace may also be specified directly
in the name part of a field specification. As a result, it is never
necessary to use the /NAMESPACE directive.
- Every field code and name in a fragment implicitly gains either the
fragment's root namespace or the current namespace. If the field
code starts with an initial dot, then the root namespace is prepended
to it, otherwise the current namespace is prepended to it. Because
the current namespace is always a subspace of the root namespace,
this means that metadata in a given fragment is never able to access
other fields outside its own root namespace.
- The exception to the above is the implicit INDEX vector, which
ignores all namespaces attached to it, either implicitly, through the
current namespace, or explicitly when specified in the metadata.
This contrasts with the behaviour of the INDEX field in the presence
of affixes, where it can appears or disappears based on the effects
of the affixes creating or modifying the specific field name "INDEX".
* Because syntactically the dot (.) now performs two functions, namely
both separating namespaces from each other and field names, but also
separating a field code from a representation suffix, there exists
ambiguity in the syntax. To resolve the ambiguity, a new representa-
tion suffix, .z has been added which does nothing. As an example, the
field code:
name.r
is interpreted as the real part of the field named "name", assuming
such a field exists. To indicate the field named "r" in the namespace
"name", the field code:
name.r.z
must be used. Note that this ambiguity only exists in a Dirfile where
both "name" and "name.r" are valid field names. If the field named
"name" doesn't exist, then the first field code, "name.r" is unambig-
uously interpreted as the field "r" in the namespace "name".
* A note on the SINDIR field: Unlike every other vector field, this
field produces character string data at the sample rate of it's input
vector, which may be surprising, in certain instances.
* GetData has supported FLAC compression since 0.9.0. Standards Version
10 now adds "flac" to the list of pre-defined encodings.
Library Changes:
* The function gd_array_len() no longer silently ignores a representation
suffix in the field_code provided. In most cases, passing a
representation suffix will now cause this function to fail with a
GD_E_BAD_CODE error.
* A couple of unnecessary malloc's have been removed from the field
code search, reducing the chance of encountering a GD_E_ALLOC error.
Notably, the functions gd_array_len(), gd_bof(), gd_entry_type(),
gd_fragment_index(), and gd_spf() will no longer produce this error at
all.
* NOTE: The GD_VECTOR_ENTRIES symbol in gd_entry_list() and gd_nentries()
calls does not match SINDIR entries, only numeric-valued vectors.
This also affects the corresponding special case functions
(gd_nvectors(), gd_vector_list() &c.)
* BUG FIX: When building in ANSI-C mode, the computation of complex-
valued RECIP fields is now correct.
* BUG FIX: The gd_include() family of functions now correctly clean up
after encountering an error. Previously, encountering a syntax error
would result in these functions erroneously adding fields specified
before the syntax error to the DIRFILE, with a bogus fragment index.
Similarly, when these functions return GD_E_REFERENCE, they no longer
add the included fragment (excluding the bad /REFERENCE directive) to
the DIRFILE without telling the caller about it.
* BUG FIX: gd_alter_protection() wasn't marking affected fragments as
modified, meaning the change in protection level would be lost upon
close unless other metadata changes were made to the fragment as well,
or a flush of the fragment's metadata was triggered explicitly with a
call to gd_rewrite_fragment().
* BUG FIX: The metadata update performed by gd_delete() now successfully
updates all fields which used the deleted field as an input.
Previously some fields could be skipped, leading to segfaults later
when these fields were accessed.
* BUG FIX: gd_add_spec() no longer creates an empty data file while
failing with GD_E_PROTECT when operating in a fragment with data
protection turned on.
* BUG FIX: gd_putdata() now refuses to write to complex-valued LINTERP
fields. Previously, the write would succeed as if the imaginary
part of the field were zero.
* BUG FIX: gd_nentries() now correctly rejects invalid values for the
type parameter.
* BUG FIX GetData now properly deals with circular series of aliases,
turning them all into dangling aliases. Previously, the alias
resolution would terminate at some arbitrary point around the loop,
resulting in internal errors arising from attempts to use an alias as
a field.
* BUG FIX: Several bugs in the I/O positioning performed before reads and
writes to compressed data have been fixed. Previously, reads and
writes could occur in the wrong place in the data stream. Reported by
S. J. Benton.
* BUG FIX: Similarly, a number of bugs associated with random-access
writes to compressed data files which were causing data corruption have
been fixed.
* BUG FIX: When reading LZMA-compressed data, gd_getdata() no longer
hangs if liblzma returns only part of a multibyte sample.
* BUG FIX: The FLAC encoding now works correctly with non-native endian
data.
* BUG FIX: Writes to ASCII (text) encoded files weren't properly updating
the file's I/O position, leading to subsequent reads and writes
occurring in the wrong place.
* BUG FIX: Trying to open a non-existent, gzip-encoded data file now
reports the correct error (GD_E_IO; "No such file or directory"),
instead of segfaulting. Reported by Matthew Petroff.
* BUG FIX: A segfault encountered when closing very large compressed
files after writing to them has been fixed.
* BUG FIX: Attempting to open a SIE-encoded data file a second time
after a first attempt failed no longer results in a segfault.
* BUG FIX: The parser no longer assumes a new string buffer returned by
the parser callback (by assigning it to pdata->line) has the size given
by pdata->buflen, which the callback is not required to update, but
instead determines the buffer size directly. Previously, this
assumption could result in a segfault within the parser.
* BUG FIX: gd_add_polynom() and gd_add_cpolynom() no longer reject valid
poly_ord values.
* BUG FIX: The gd_[m]add() family of functions are now better at
rejecting invalid data types.
* BUG FIX: A segfault-on-error has been fixed in gd_[m]alter_spec.
* BUG FIX: The gd_madd() functions longer accept aliases as parent field
names.
* BUG FIX: Setting n_fields (for LINCOMs) or poly_ord (for POLYNOMs) to
an out-of-range value and then calling gd_free_entry_strings() no
longer results in a segfault.
* BUG FIX: A rare segfault has been fixed in gd_carrays() and
gd_strings().
API Changes:
* The new function gd_alloc_funcs() allows callers to change the memory
manager used by GetData to allocate and de-allocate heap buffers that
it returns. The functions gd_entry(), gd_error_string(),
gd_fragment_affixes(), gd_linterp_tablename(), and gd_raw_filename()
will use this memory manager to allocate the buffers they return. The
function gd_free_entry_strings() will use this memory manager to free
strings. Even if an alternate memory manager is specified, GetData
will still use the Standard Library's malloc() and free() for much of
it's internal storage.
* A new function, gd_match_entries(), extends the functionality of
gd_entry_list() and gd_nentries() by additionally allowing both
searches restricted to a particular fragment and also regular
expression matching against entry names.
* A number of functions which used to return -1 on error now instead
return an appropriate error code, previously only available through
gd_error(). These error codes are all negative-valued. Functions
whose error returns have changed and now do this are:
gd_add(), gd_add_alias(), all the gd_add_<entry_type>() functions,
gd_add_spec(), gd_alter_affixes(), all the gd_alter_<entry_type>(),
functions, gd_alter_encoding(), gd_alter_endianness(),
gd_alter_entry(), gd_alter_frameoffset(), gd_alter_protection(),
gd_alter_spec(), gd_array_len(), gd_bof(), gd_delete(), gd_desync(),
gd_dirfile_standards(), gd_discard(), gd_entry(), gd_entry_type(),
gd_eof(), gd_flush(), gd_fragment_affixes(), gd_fragment_index(),
gd_frameoffset(), gd_get_carray(), gd_get_carray_slice(),
gd_get_constant(), gd_hidden(), gd_hide(), gd_include(),
gd_include_affix(), gd_include_ns(), gd_madd(), gd_madd_alias(),
all the gd_madd_<entry_type>() functions, gd_madd_spec(),
gd_metaflush(), gd_move(), gd_nframes(), gd_open(),
gd_parent_fragment(), gd_protection(), gd_put_carray(),
gd_put_carray_slice(), gd_put_const(), gd_raw_close(),
gd_rename(), gd_rewrite_fragment(), gd_seek(), gd_spf(), gd_sync(),
gd_tell(), gd_unhide(), gd_uninclude(), gd_validate(),
gd_verbose_prefix()
* gd_add_indir(), gd_add_sarray(), gd_add_sindir(), gd_madd_indir(),
gd_madd_sarray(), gd_madd_sindir(), gd_alter_indir(),
gd_alter_sarray(), and gd_alter_sindir() have been added to manipulate
metadata of the new field types.
* gd_msarrays(), gd_get_sarray(), gd_get_sarray_slice(), gd_put_sarray(),
gd_put_sarray_slice(), gd_sarrays() have been added to read and write
SARRAY values.
* gd_fragment_namespace() and gd_include_ns() have been added to read and
write the root namespace of a fragment.
* gd_put_string() now returns zero on success and a negative-valued error
code on error, as well, following the lead of gd_put_constant() and
gd_put_sarray(). Also note that the return type is now int, where
previously it was size_t, despite the documentation having always
claimed it returned int.
* A new data type symbol, GD_STRING, exists to represent string data.
The gd_native_type() function now returns GD_STRING for STRING fields
(as well as for the new SARRAY and SINDIR fields).
* The GD_FUNCTION_ALIASES block, and hence the long-deprecated
GetData-0.6 API, has been removed from getdata.h. Anyone still using
the GetData-0.6 API should modernise to avoid known problems with that
API.
* The gd_shift_t type, which was used for PHASE field shifts, has been
deprecated. It has been replaced with gd_int64_t, which is what it
was always typedef'd to.
* BUG FIX: gd_getdata() and gd_putdata() now properly report GD_E_IO when
an I/O error occurs while reading a LINTERP table. Previously GD_E_LUT
would be erroneously returned, as if the table file were empty.
Bindings Changes:
* MATLAB: The FLAGS argument to GD_INCLUDE is now optional and defaults
to zero.
* C++, F77, F95, PERL: The bindings for gd_put_string() have changed to
reflect changes in the C library. Fortran 77 no longer returns a
n_wrote integer; Fortran 95 implements fgd_put_string a subroutine;
C++ and Perl bindings now return integer zero on success, and negative
on error.
* IDL, MATLAB, PHP BUG FIX: Bindings for the C functions gd_raw_filename
and gd_linterp_tablename now no longer leak the string returned by the
C API.
* PYTHON BUG FIX: A UnicodeEncodeError while assigning to
dirfile.verbose_prefix no longer results in that attribute being set
to None. Instead, it retains its former value, which reflects what
actually happens in the underlying C library in this case.
* PYTHON BUG FIX: Objects returned by dirfile.entry() and
dirfile.fragment() weren't being initialised with the correct reference
count, leading to memory leaks when they went out of scope. Reported
by Alexandra Rahlin.
Miscellaneous:
* The --enable-assert configure option, which hasn't done anything for a
long time, has been removed.
* A new configure option, --disable-util, can be used to suppress
building of the executables in the util/ subdirectory.
* In the standard autotools build system, encodings which use external
libraries for compression (gzip, bzip2, flac, lzma, slim, zzip, zzslim)
are now by default built as dynamically loaded modules. To recover the
old build behaviour, which put everything into the core GetData library
binary, pass --disable-modules to ./configure. Using modules intro-
duces a runtime dependency on GNU libltdl. The CMake-based build
system used in the native Microsoft Windows source release retains the
old (monolithic) behaviour.
|=========================================================================|
New in version 0.9.4:
Library Changes:
* BUG FIX: Arbitrarily-long reads of FLAC-encoded files now work.
Previously, each FLAC frame was written to the start of the output
buffer, overwriting the previous frame, and leaving most of the
buffer uninitialised. Reported by S. J. Benton.
* BUG FIX: Data read from FLAC-encoded 1-byte types are now correct.
Previously, although all data requested was read, only the first half
would be returned, encoded as 16-bit data (i.e., with a zero-byte
between every sample).
Bindings Changes:
* PYTHON BUG FIX: An erroneous preprocessor definition which made the
Python bindings unbuildable for Python 3 has been fixed. Reported by
Akito Kusaka.
Miscellaneous:
* A work-around has been implemented in the configure script to avoid a
bug in bash 4.1's parser. Reported by Akito Kusaka.
* The way configure decides the Python module path has changed slightly
to provide a better default under debian-based systems when using
debhelper. Reported by S. J. Benton.
|=========================================================================|
New in version 0.9.3:
Library Changes:
* GetData can now read SIE files containing the optional nine-byte header
(which can be created by daisie). When read by GetData, information in
the header is completely ignored (because the GetData metadata contains
all the necessary information). GetData never writes the header, but
calls to gd_putdata() will preserve an existing header. Calls which
re-code the file (like gd_alter_endianness(), gd_alter_raw(), &c.) will
result in an existing header being deleted.
* BUG FIX: Filenames and line numbers appearing in GD_E_FORMAT error
strings returned by gd_error_string() are correct again. This bug
also affected parser metadata sent to a registered parser callback.
* BUG FIX: On platforms where char is signed, the library no longer
rejects field names containing bytes with the top bit set.
API Changes:
* gd_verbose_prefix() can now be used on invalid dirfiles. Previously,
this function would return GD_E_BAD_DIRFILE when passed an invalid
DIRFILE pointer.
* BUG FIX: When trying to access a LINTERP table file in a non-existent
directory, GetData now reports the correct error (No such file or
directory). Reported by Johanna Nagy.
Bindings Changes:
* PHP: PHP7 support has been added.
* PYTHON: Python3 support has been added based on a patch from Matthew
Petroff. The earliest supported Python3 version is 3.2. Python2 is
still supported from version 2.4, but Unicode support must be enabled
in Python2. Support for Python 2.3 has been dropped.
* PYTHON: Under Python3, the bindings run into the issue of GetData not
knowing the character encoding of Dirfile metadata (the C library
just deals with bytes). As a result, under Python3, by default,
most strings returned by the library are returned as encoded bytes()
objects, instead of native Unicode str() objects. (This is true
under Python2 as well, but less obvious since the native Python2
str() object is encoded.)
To help deal with this, dirfile and entry objects now have a
character_encoding attribute which can be set to inform pygetdata of
the character encoding to use to decode the strings returned by the
C library into Unicode strings. There is also a global
pygetdata.character_encoding object which can be used to set the
default encoding for newly-created pygetdata objects. See the module
documentation for details. The default for character_encoding is None,
implying no decoding should occur. In Python3, paths are handled
separately (since the filesystem encoding may be different than the
GetData metadata encoding). Error strings are decoded if possible, and
then ASCII encoded to ensure they're always available, regardless of
the capabilities of standard error.
* PYTHON: As a side-effect to the above, pygetdata now accepts Unicode
strings (both in Python2 and in Python3). The specified
character_encoding will be used to encode them to C strings before
being passed to the C library. If no character_encoding is specified,
the current locale's default encoding will be used.
* PYTHON: When using Python 2.6 or newer, calling repr() on a
pygetdata.entry object now returns an eval()able string.
* PYTHON: When using Python 2.7 or newer, a very small C API is produced
for pygetdata. It is defined in pygetdata.h installed alongside the
pygetdata module. This C API is needed by daisie, the stand-alone SIE
encoding library.
* C++ BUG FIX: Including another GetData header before getdata/dirfile.h
no longer results in a fatal circular dependency.
* PHP BUG FIX: The numbits parameter to gd_madd_sbit() is now optional
and defaults to 1, as with other similar functions.
* PHP BUG FIX: Fixed a memory leak on error in gd_getdata().
* PYTHON BUG FIX: Calling pygetdata.dirfile.[m]carrays no longer crashes
if return_type=pygetdata.NULL. In this case, None is returned in place
of the data arrays.
* PYTHON BUG FIX: Entry objects returned by dirfile.entry() now properly
represent scalar field codes.
* PYTHON BUG FIX: An out-of-memory condition encountered by the
underlying C library is now reported using the standard MemoryError
exception. Previously, the bindings would attempt to instantiate a new
pygetdata.AllocError exception when this happened, which wouldn't work
if no memory was available. (The CPython interpreter pre-allocates an
instance of MemoryError to deal with this situation.) For backwards
compatibility, pygetdata.AllocError is now an alias for MemoryError.
* PYTHON BUG FIX: a scalar field code string can now be assigned to the
threshold attribute of WINDOW entries.
* PYTHON BUG FIX: Attempting to delete various attributes from
pygetdata objects now either succeeds or else raises an exception,
instead of crashing.
Miscellaneous:
* The default install directory for the IDL, Perl, PHP, and Python
bindings has changed: they are now all installed under ${exec_prefix}
when possible. The install directories can still be overridden by
./configure options.
|=========================================================================|
New in version 0.9.2.1:
Miscellaneous:
* This release fixes one entry in the test suite (alter_entry_scalar3r)
which was broken in the original 0.9.2 release. The library and
bindings are unchanged (and report their version to be simply 0.9.2).
The error in the test suite was reported by Dinar Valeev.
New in version 0.9.2:
Library Changes:
* BUG FIX: When using a FLAC-encoded reference field, gd_nframes() no
longer leaks file descriptors.
* BUG FIX: A segfault in gd_entry_list() introduced in 0.9.1 has been
fixed. Reported by Christian Trippe.
* BUG FIX: A descriptor leak on error has been plugged in gd_desync().
* BUG FIX: A segfault triggered by encountering an I/O error while
writing metadata has been fixed.
* BUG FIX: Memory leaks in the FLAC and ZZIP encodings, plus single-byte
leaks in gd_add_string() and gd_madd_string() have been plugged.
* BUG FIX: gd_alter_entry() and gd_[m]alter_spec() no longer mangle the
values of entry parameters when asked to remove scalar field codes.
Reported by Dan Horák.
Bindings Changes:
* PYTHON BUG FIX: Fixed a potential segfault in dirfile.get_carray() on
platforms where sizeof size_t != sizeof int. Reported by Dan Horák.
* PHP BUG FIX: Calling gd_discard or gd_close on a persistent dirfile now
does nothing, instead of closing the Dirfile and corrupting the
persistent resource.
|=========================================================================|
New in version 0.9.1:
Library Changes:
* Functions returning entry lists (gd_entry_list(), &c.) or bulk scalar
field data (gd_constants(), gd_carrays(), &c.) no longer run through
the entry list twice, resulting in noticeable speed improvements when
operating on large dirfiles.
* BUG FIX: Top-level aliases pointing to metafields, and non-hidden
aliases pointing to hidden entries are now included in the data
returned by gd_nentries() and gd_entry_list() when they should be.
* BUG FIX: Surprising combinations of hidden fields, meta fields, and
aliases no long confuse the functions which return bulk scalar data
(gd_constants, gd_strings, gd_carrays, &c.). Previously, given the
right metadata, these functions would segfault, return incorrect
data, or raise GD_E_INTERNAL_ERROR.
* BUG FIX: The functions gd_delete(), gd_hide(), gd_hidden(), and
gd_unhide() no longer segfault when asked to operate on a field that
doesn't exist; instead, they properly report error (GD_E_BAD_CODE).
* BUG FIX: gd_seek() and gd_tell() now always return -1 on error.
Previously, they would return zero for some errors; they always
still correctly set the dirfile error.
* BUG FIX: Several memory or other resource leaks triggered by error
returns have been fixed.
* BUG FIX: GetData no longer ignores errors reported by zlib when
writing to gzipped data files. In the past, this could, in some
cases, cause GetData to wedge.
API Changes:
* BUG FIX: gd_constants() and gd_mconstants() now return NULL and set the
dirfile error to GD_E_BAD_TYPE if passed GD_NULL as the return_type.
Previously these functions would return NULL without setting an error
when passed GD_NULL.
* BUG FIX: gd_getdata() and gd_putdata() now truncate their operation to
avoid overflow. If size_t is N-bytes wide, and GetData is operating on
an M-byte wide data type, then truncation occurs at 2**(8N-M)-1
samples. Note, however, that truncation can also be triggered by
intermediate data products, which may have wider data types (larger M)
than the type specified by the caller. If truncation occurs, a short
read/write will occur, without raising an error. Truncation occurs
even when the data type is GD_NULL (i.e. when these functions have
nothing to do). For GD_NULL, M is taken to be 1. Callers should
check the return value of these functions, and make another call to
handle the truncation, if necessary.
* BUG FIX: Instead of seeking to a random place, gd_seek() now returns
GD_E_RANGE when the specified offset, or the resultant position would
overflow a 64-bit signed integer. Note, however, that, in most cases,
the file size limit imposed by your filesystem or OS will be
encountered first (in which case GD_E_IO will be returned). For
comparison, the Windows NTFS driver limits files to 2**44 bytes,
as does the ext4 filesystem.
* BUG FIX: Similarly, gd_getdata() and gd_putdata() return GD_E_RANGE if
asked to operate on data outside the addressable range (more than 2**63
samples past the start of the dirfile).
Bindings Changes:
* PYTHON: The return_type parameter to dirfile.get_constant() and
dirfile.get_carray() is now optional, and will default to the native
type of the data returned, as happens with dirfile.getdata().
* IDL BUG FIX: Using !GD.NULL with GD_GETDATA(), GD_GET_CONSTANT() and
GD_GET_CARRAY() now works. In this case GD_GETDATA() returns the
number of samples read (as in the C API), and the other functions
return zero on success.
* MATLAB BUG FIX: Using GD.NULL with gd_getdata(), gd_get_constant(),
gd_get_carray(), and gd_get_carray_slice(), now works. In this case,
gd_getdata() returns the number of samples read (as in the C API), and
the other functions return zero on success.
* PERL BUG FIX: Using $GetData::NULL with getdata(), get_constant(),
get_carray(), and get_carray_slice() now works. In scalar context,
getdata() returns the number of samples read, while the others return
undef. In list context, they all return an empty array in this case.
* PHP BUG FIX: Passing data with a GD_NULL data type now behaves as
expected (i.e. the passed data is ignored).
* PHP BUG FIX: Using GD_NULL with gd_getdata(), gd_get_constant(),
gd_get_carray() now works. In this case, gd_getdata() returns the
number of samples read (as in the C API), and the other functions
return TRUE on success.
* PYTHON BUG FIX: Using return_type=pygetdata.NULL with
dirfile.getdata(), dirfile.get_constant() or dirfile.get_carray() now
works. dirfile.getdata() will return the number of samples read (as in
the C API), ignoring as_list. The other functions return None on
success.
* PYTHON BUG FIX: Reflecting the merge of GD_E_BAD_REPR into
GD_E_BAD_CODE in the C API in 0.9.0, the BadRepr exception is now an
alias of BadCode.
Miscellaneous:
* The build system now uses ExtUtils::MakeMaker instead of Module::Build
to build the perl bindings. Module::Build was removed from the Perl 5
core in Perl 5.22. ExtUtils::MakeMaker is still a core module. The
bindings are unchanged; the change in build prerequisites is the only
difference users should notice due to this switch.
* A new configure option, --disable-large-tests, will cause the test
suite to skip running tests that write large amounts (>~10MB) of data.
|=========================================================================|
New in version 0.9.0:
Library Changes:
* Literals in format metadata may now have complex form (i.e. include a
semicolon) when the parameter is purely real. However, a non-zero
imaginary part is still an error.
* gd_free_entry_strings() now NULLs pointers after freeing them.
* gd_entry() now returns entry metadata when they contain scalar field
codes which do not exist. In this case the GD_EN_CALC flag in the
object will not be set. Previously, on such entries, this function
would fail with the error GD_E_BAD_SCALAR, and return nothing.
* gd_rename() now by default updates the target of ALIASes pointing to a
renamed field to point to the new field instead of leaving them dangle.
(But see GD_REN_DANGLE in the API section below).
* CARRAYs are no longer truncated to GD_MAX_CARRAY_LENGTH elements.
Flushing metadata to disk will now fail if writing a CARRAY would
overflow a format file line. (It's platform specific, but format file
lines are typically permitted to be at least 2**31 bytes long, so such
an error usually indicates something pathological happening.) The
GD_MAX_CARRAY_LENGTH symbol has been removed from the GetData header
file.
* Write support for bzip2-encoded and lzma-encoded data has been added.
LZMA write support is only available for .xz files, not the obsolete
.lzma format. The write support occurs out-of-place, just like how
writing gzip-encoded data works. See the gzip discussion in the 0.8.0
section below for important notes.
* A new encoding scheme using the Free Lossless Audio Codec (FLAC) to
compress data has been implemented. For some datasets, it provides a
good trade-off between speed and compression. Like gzip, bzip2, and
lzma, is also uses out-of-place writes (see previous point).
* A newly-created dirfile is now always opened in read-write mode, ignor-
ing the access mode specified in the call. Previously, specifying both
GD_RDONLY and GD_CREAT in open calls would result in an access mode
(GD_E_ACCMODE) error if the dirfile didn't already exist.
* Many functions which used to silently ignore representation suffixes in
field codes passed to them no longer do that. Most of these will
report an error (GD_E_BAD_CODE) if passed a representation suffix. The
affected functions are: gd_bof, gd_entry, gd_entry_type, gd_eof,
gd_flush, gd_linterp_tablename, gd_put_carray, gd_put_carray_slice,
gd_putdata, gd_raw_close, gd_raw_filename, gd_seek, gd_spf, gd_sync,
gd_tell.
* The error code GD_E_BAD_REPR has been merged into GD_E_BAD_CODE. The
symbol GD_E_BAD_REPR remains as an alias for GD_E_BAD_CODE, but is
deprecated.
* Attempts to seek past the end-of-field with gd_seek() now always
succeed, although the resultant position is encoding specific.
Previously, attempting to seek past the end-of-field on some encodings
would return an error.
* BUG FIX: The library now properly recovers from an I/O error while
trying to open an unencoded datafile. Previously, such an error would
poison the library's bookkeeping data, preventing all subsequent
attempts to open that file unless the Dirfile was re-opened. Reported
by Alexandra Rahlin.
* BUG FIX: GetData no longer segfaults when trying to do a large forward
seek before a write to a gzipped file. Reported by Joy Didier.
* BUG FIX: gd_putdata() no longer ignores I/O errors while seeking to the
first sample of a write.
* BUG FIX: If the reference field is being written to, gd_nframes() now
flushes it first before calculating the size of the dirfile.
Previously a short count could result for some encodings in this case.
* BUG FIX: Calling gd_putdata() to write gzip data with a non-zero
starting offset equal to the field's current I/O position, no longer
result in the call hanging.
* BUG FIX: In addition to the addition of write support mentioned above,
a number of problems with reading LZMA files has been fixed, which
should result in fewer segmentation faults.
* BUG FIX: The parser no longer silently appends a closing > to scalar
field codes that contain an unmatched opening < (e.g. "scalar<3").
This is now interpreted as a simple field code (which may be rejected
later due to the presence of the invalid '<' character).
* BUG FIX: The parser no longer interprets various numbers as field codes
when it shouldn't (e.g. when specifying a PHASE shift as "1." instead
of "1").
* BUG FIX: When writing scalar field codes to disk which could be inter-
preted as a number (e.g. the field code "1"), the library now forces
the interpretation of these field codes as codes rather than numbers by
appending a scalar index (making, e.g., "1<0>"), which is harmless.
Previously, these were written as-is, resulting in misinterpretation
the next time the Dirfile was opened. This only happens with Standards
Version 8 or later, see the following for earlier versions.
* BUG FIX: If the current Standards Version in effect is 7 or earlier,
ambiguous field codes (e.g., "1"), are now rejected by gd_[m]add() and
gd_alter_entry() with the error GD_E_BAD_CODE, since they can't be
represented in the metadata on disk. For the behaviour with later
Versions, and in permissive mode, see the previous.
* BUG FIX: If performing a metadata update due to renaming fields
(perhaps by passing GD_REN_UPDB to gd_rename()) results in an invalid
field code due to affix restrictions, the update now fails (but see
GD_REN_FORCE). Previously the invalid field code would be stored,
leading to errors when flushing the modified metadata to disk.
* BUG FIX: When performing a metadata update due to a renamed field, the
field codes containing subfields of the renamed field are now also
updated, including field codes specifying meta subfields which do not
exist.
* BUG FIX: reading a LINTERP table with fewer than two lines no longer
results in a segfault on close/discard.
* BUG FIX: gd_alter_raw() and similar no longer fail when asked to re-
encode the data file of a RAW field which has not been previously
accessed.
* BUG FIX: A previously-read LINTERP table is now always discarded when
changing table paths with gd_alter_linterp() or similar. Previously
these obsolete, cached LUTs would sometimes linger, causing incorrect
LINTERP computation.
* BUG FIX: The I/O position reported by gd_tell and gd_seek for slim,
zzip, and zzslim encoded data is now correct.
API Changes:
* CLARIFICATION: The macro GD_SIZE() declared in getdata.h is indeed part
of the public API. It returns the size in bytes of a sample of data of
a given type (e.g. GD_SIZE(GD_COMPLEX64) returns 8). It has been
around since GetData-0.3.0, but has only been documented since
GetData-0.8.3.
* The comp_scal member of the gd_entry_t object has been replaced with a
flags member, containing a flag (GD_EN_COMPSCAL) with the meaning of
the former comp_scal member. There are also flags for hiddenness
(GD_EN_HIDDEN) and whether the scalar entry codes in the field defi-
nition have been dereferenced (GD_EN_CALC).
* gd_[m]add() and gd_alter_entry() can now be used to set or change the
hiddenness of a field by setting or clearing the GD_EN_HIDDEN bit in
the supplied gd_entry_t object.
* Two new rename flags have been added:
- GD_REN_DANGLE which indicates the library shouldn't update ALIASes
whose target has been renamed (instead it will turn them into
dangling aliases)
- GD_REN_FORCE which causes the library to skip updating field codes
which would be invalid due to affixes instead of failing.
* The move_data argument of gd_move() has been replaced with a flags
argument which accepts the GD_REN_* flags, which have the same meaning
as they do with gd_rename().
* gd_move_alias() and gd_delete_alias() have been deleted: their
functions are now performed by gd_move() and gd_delete(), which now
operate on the alias itself when given the field code to an alias,
rather than the field the alias points to.
* A number of different error codes which indicated the same problem (an
I/O error returned by the operating system) have been merged into one.
The error codes GD_E_OPEN, GD_E_TRUNC, GD_E_RAW_IO, GD_E_OPEN_FRAGMENT,
GD_E_FLUSH are replaced by the new error GD_E_IO. The old symbols
remain as aliases but are deprecated. The corresponding error strings
also now include information from the underlying encoding library,
where possible. There is one exception to this merge: attempts to
flush metadata lines which are too long are now reported using
GD_E_LINE_TOO_LONG. Previously, these errors used GD_E_FLUSH.
* The error code GD_E_OPEN_LINFILE has also been removed. It has been
split into two parts:
- I/O errors resulting from reading the LINTERP table file are now
reported using GD_E_IO;
- Syntax errors in the table are reported using the new GD_E_LUT error
code. GD_E_OPEN_LINFILE remains as a deprecated alias for GD_E_LUT.
* gd_encoding_support() has been added to permit run-time determination
of supported encodings.
* gd_array_len() is the new name for gd_carray_len(). It now also
handles STRINGs (which have a length of one). The gd_carray_len() name
remains in the library, but has been marked deprecated.
* BUG FIX: If the dirfile path provided cannot be resolved (due to, for
instance, a symbolic link pointing to a non-existent path), gd_open()
and friends now return the correct error code (GD_E_IO).
* BUG FIX: gd_naliases() now returns an unsigned int, and zero on error,
as documented.
* BUG FIX: The API on 32-bit systems, which was broken in 0.8.7 and only
partially fixed in 0.8.8, should now work as expected again.
Bindings Changes:
* PHP bindings have been added.
* C++: There is no longer a default value for the "index" argument for
Entry methods (including subclasses) which accept it (viz. Input,
Scalar, ScalarIndex, Scale, CScale, Offset COffset, Coefficient,
CCoefficient). The exception to this is with Entry subclasses for
which zero is the only allowed value for the parameter.
* F77 and F95: The bindings no longer raise SIGABRT when the dirfile
space is exhausted. Instead they simply return a dirfile unit number
referencing a static, invalid dirfile.
* F77: Functions to add fields with named scalar parameters have been
added (GDASBT GDASCL GDASCP GDASCR GDASLC GDASMX GDASPH GDASPN GDASRC
GDASRW GDASSB GDASWD), but only for those field types which permit
named scalars. Similarly, functions for altering field metadata with
named scalars are also present (GDLSBT GDLSCL GDLSCP GDLSCR GDLSLC
GDLSMX GDLSPH GDLSPN GDLSRC GDLSRW GDLSSB GDLSWD). These are provided
as an alternative to using GDASCA after the fact.
* IDL: The entry structure parser has been rewritten. It no longer
requires members which it doesn't need, and is also a lot more lax
about numerical data types. Notably, it now ignores a supplied
COMP_SCAL member. Floating point parameters can be specified in either
the base name (M, B, A, DIVIDEND) or else the member prefixed with 'C'
(CM, CB, CA, CDIVIDEND), whatever numerical type. The bindings will
ingest them appropriately. Also, N_FIELDS and POLY_ORD, may be
omitted, and will be calculated from the supplied data. A scalar
IN_FIELDS is treated like an single element array.
* IDL: GD_REFERENCE is now a function, instead of a procedure, as the
documentation has always claimed it was. It returns the current
reference field (or the empty string, if there is none). The second
parameter, the new reference field, is optional. (Previously the
second parameter was required.)
* PERL: The entry hash parser has been rewritten. It no longer requires
keys which it doesn't need.
* PERL: alter_entry() now only updates defined elements in the passed
entry hash.
* PYTHON: Building the python bindings now requires NumPy. Previously,
NumPy support was optional.
* PYTHON: for backwards compatibility, exceptions now exist for
deprecated error codes (such as OpenError). These deprecated
exceptions are simply aliases for the current ones and are never
returned by the bindings.
* C++ BUG FIX: The Entry methods Input, Scalar, and ScalarIndex
(including subclasses) now return zero or NULL when passed an out-of-
range index value. Previously they would return, variously, zero,
NULL, another value for some other, valid index value, or segfault.
* C++ BUG FIX: The flags parameter to Dirfile::Delete() is now unsigned,
as it is in the C API.
* F95 BUG FIX: fgd_add and fgd_alter_entry no longer ignore named scalar
parameters provided in supplied entry structures.
* PYTHON BUG FIX: Several memory leaks have been plugged. Patch from
Matthew Petroff.
Miscellaneous:
* The minimum autotools versions have been bumped. Autoconf-2.65 or
newer, automake-1.13 or newer, and libtool-2.2.7b or newer are now
required to rebuild the configure script and associated build environ-
ment. NOTE: In general, most people building GetData from a source
release don't need the tools to build GetData; the autotools are only
needed if changes need to be made to the configure script or Makefile
input files provided in the release or if building from the repository.
|=========================================================================|
New in version 0.8.9:
Library Changes:
* BUG FIX: The metadata writer now correctly stores bytes in the range
0x01 through 0x1F to the format files (encoded as hex escape sequences:
\x##).