v4.0

New stable release! 🚀 🎉
There are new features, bug fixes but also some architecture changes and a new documentation website 🤩
The core concept of the framework is the same but there are some breaking changes in APIs and technology replacements (plugins API).

These changes have been extensively tested in the new SceneGate projects (UI tool, Ekona, Lemon) and modding projects. Releasing v4.0 is a new milestone in SceneGate that allows us to start shipping the first versions of these projects.

Summary

• 📚 Documentation! checkout in the project site
• 🔢 Enhancements in the IO namespace bringing compatibility with the native .NET Stream type
• ♻️ New converters API that allows initialization via standard class constructors
• 📋 Cloneable formats
• 📃 New simple base class for custom encoding implementations
• 🔌 New native and safer plugin API. No longer based on MEF.
• 🆕 .NET 6.0 and 8.0 support with nullable annotations. .NET Framework is not supported anymore.

Breaking changes

• Drop support of .NET Framework
• Rename library Yarhl.Media to Yarhl.Media.Text
• Plugin API moved to new library Yarhl.Plugins
• Remove APIs ConvertTo, TransformTo and DataStream.ReadFormat<T>()
• Obsolete IInitializer<T>. Use the constructor instead.
• DataStream does NOT throw EndOfStreamException when reading bytes.
  • ReadByte returns -1 when it reaches the end instead of throwing an exception.
  • DataReader keeps the behavior and it will throw EndOfStreamException when reading any data type.
• Move DataStream.Length setter to its own method SetLength(long).
• Rename TextReader and TextWriter to TextDataReader and TextDataWriter.
• Removed IStream interface in favor of Stream.
• TextDataReader will not return null string if it starts reading from the end already. It will throw EndOfStreamException.
• DataStream.Seek now requires a negative argument to go back positions when origin is SeekOrigin.End.
• Binary object (de)serialization removed from DataReader and DataWriter and moved into new types BinaryDeserializer and BinarySerializer.
  • The attribute BinarySerializable is no longer required and it's removed.
  • The attribute BinaryOrder is mandatory for properties in projects running in .NET 6.0
  • The attribute property ReadAs and WriteAs is now UnderlyingType
  • The attribute for enums is optional. Its defined underlying type will be used.

As part of this release we had 49 issues closed.

What's Changed

• Formats and converters
  • ✨🔥 Implement new converter API by @pleonex in #191
  • ✨ Implement ICloneableFormat by @Kaplas80 in #168
  • ✨ Enhance debugging view by @pleonex in #209
• IO
  • ✨ Refactor and improvements in binary object (de)serialization by @pleonex in #208
  • ✨ Implement DataStream inheritance with Stream and support of Readers and Writers with Stream by @pleonex in #161
  • 👕 Rename TextReader and TextWriter to TextDataReader and TextDataWriter by @pleonex in #162
  • ✨🔥 Add Stream argument constructor to DataStream and remove IStream interface by @pleonex in #170
  • ✨ Implement slice for DataStream by @pleonex in #204
  • ✨ Implement factories to create DataStreams and Nodes from array and standard streams by @pleonex in #173
  • 🐎 Improve memory consumption of DataStream.WriteTo by @pleonex in #183
  • ✨ Use DataStream type in Stream properties for readers and writers and explicitly set CanTimeout to false by @pleonex in #172
  • ✨🐛 Stream overloads in BinaryFormat and NodeFactory and fix threading issue by @pleonex in #163
  • ✨ Implement constructor to create BinaryFormat from a file by @pleonex in #205
  • 🐛 Fix behavior of DataStream.Seek when SeekOrigin is End by @Kaplas80 in #177
  • 🐛 Comparing stream to itself returns false by @pleonex in #199
  • 🐛 WriteTo truncates output file by @pleonex in #203
• FileSystem
  • ✨ Implement node renaming by @pleonex in #200
  • ✨ Implement extension methods to transform node collections by @pleonex in #201
  • ✨ ChangeFormat fluent style returning itself by @pleonex in #202
  • ✨ Advanced filter in FromDirectory by @Kaplas80 in #166
  • ✨ Support opening nodes from Windows symlinks by @pleonex in #180
  • 🐛 Set FileOpenMode at NodeFactory.FromFile by @Kaplas80 in #165
  • 🐛 Fix NodeFactory.FromDirectory by @Kaplas80 in #188
• Media.Text
  • 👕 Rename Yarhl.Media to Yarhl.Media.Text by @pleonex in #187
  • ✨ Implement simple base class for custom encoding implementations by @pleonex in #181
  • 🐛 Fix Binary2Po converter not reading from the beginning of the stream by @Kaplas80 in #169
• Plugins
  • ❇️ Re-implementation of plugins API by @pleonex in #206
• Documentation
  • ✨📚 New documentation website and project name re-meaning by @pleonex in #192
  • 🐛 Fix "Yarhl in a nutshell" link by @Timo654 in #182
  • 🐛 Fixed a small error in the Docs by @priverop in #184
• Build system
  • ✨ Annotate API reference types for nullable awareness by @pleonex in #176
  • 🔥 Deprecate .NET Framework by @pleonex in #178
  • ✨ Support .NET 8.0 by @pleonex in #198
  • ✨ Update build system to build and test for .NET 6 by @pleonex in #174
  • ✨ New build system and warning clean-up by @pleonex in #197
  • ⬆️ Bump build system and use Cake v2.1 by @pleonex in #179
  • ⬆️ Migrate to Cake 3.0 and latest .NET SDK by @pleonex in #190
  • ⬆️ Update dependencies and fix code coverage by @pleonex in #171
  • 🔧 Update build system for Ubuntu 20.04 by @pleonex in #167
  • 🔧 Disable auto-comment on release by @pleonex in #160

Full Changelog: v3.1.0...v4.0.0

v3.1.0

As part of this release we had 21 issues closed.
Several features and bug improvements in IO like the new cool binary (de)serializer. Use the new build system from PleOps.Cake. Deprecate .NET Framework 4.6.1.

Special thanks to @Kaplas80, @Megaflan, @Darkmet98 and @pleonex.

Bugs

• #147 DataStream.WriteTo does not create the file when the length is zero
• #146 Remove double slashes in path
• #141 Fix multi-line support in ExtractedComment of PO files
• #140 Extracted comments with multi lines is splitted by space instead of new line on Binary2Po
• #139 NodeFactory fails to create node tree when the path contains double slashes
• #136 DataStream.WriteTo does not create the file when the length is zero
• #135 DataStreamFactory does not check if the file exists when opening for reading

Enhancements

• #155 Migrate build system to PleOps.Cake and deprecate .NET Framework 4.6.1
• #154 Support .NET 5.0
• #153 Support .NET 5
• #152 Support Int24 type in binary (de)serializer
• #151 Implement binary (de)serializer from objects
• #149 Custom string terminators
• #148 Solved DataStreamFactory does not check if the file exists when openi…
• #145 Support float and double types in ReadByType and WriteOfType
• #144 Implement Stream.WriteTo with custom offset
• #123 Improve build and release process
• #73 Implement Stream.WriteTo with custom offset

Yarhl 3.0

This release focuses in API user-experience enhancements. The transform and format API methods and base classes are now easier to understand and use. New convenience methods introduced and factories. Extensibilities in the IO namespace to support advance use cases as we have more applications using Yarhl.

Special thanks to all the contributors: @Kaplas80, @pleonex and @priverop and to all our users providing great feedback. Especially the guys from TraduSquare.

Features

• New API to pass parameters to converters: IInitializer<T> (#93 by @pleonex)
• New API DataStreamFactory to create DataStream (#111 by @pleonex)
• New API to keep ownership of underlying streams (#118 by @pleonex)
• New API to create nodes from a part of a stream: NodeFactory.FromSubstream (673ad2d by @pleonex)
• New API to create DataStream from a byte array (9158dea by @pleonex)
• Thread-safe DataStream for sharing an IStream (#129 by @Kaplas80)
• Add extensibility to streams with new interface IStream (#109 by @pleonex)
• Lazy open files to avoid hitting maximum file handlers opening a folder (#111 by @pleonex)
• Improve memory and performance by using RecyclableMemoryStream for memory DataStream (#111 by @pleonex)
• New interface IBinary to mark binary types (other than BinaryFormat) (0450b98 by @pleonex)
• Improve performance of DataStream.Compare (#110 by @pleonex)
• New API DataReader.ReadStringToToken() (#137 by @pleonex)
• New constructor in TextReader and TextWriter with the encoding name (#114 by @pleonex)
• Auto-register additional encodings of .NET Core (#114 by @pleonex)
• New API to sort children (#131 by @Kaplas80)
• New node tags FileInfo and DirectoryInfo in nodes from the NodeFactory (#128 by @Kaplas80)
• Optional dispose of nodes when removing children (#130 by @Kaplas80)
• Enhancements to NodeFileContainer.MoveChildrenTo for merging nodes instead of replacing (#130 by @Kaplas80)
• Support relative path searches (#99 by @pleonex)
• New API to remove nodes (#100 by @pleonex)
• Prevent adding as a child a parent node (#115 #133 by @pleonex and @Kaplas80)
• New cookbook with small code snippets (#108 by @priverop)
• New CII Best Practices badge (6e06654 by @pleonex)

Breaking changes

• Change license from GPL v3 to MIT which is more permissive (#121 by @pleonex)
• Split PO converters. Binary format into PO is a new converter: Binary2Po (#120 by @pleonex)
• Change Node.Transform methods into Node.TransformTo and Node.TransformWith (#93 by @pleonex)
• Move static converter methods from Format to new static class ConvertFormat (#92 by @pleonex)
• Convert Format into an empty interface IFormat (#92 by @pleonex)
• Node.Format property is now getter-only. New method added Node.ChangeFormat (#92 by @pleonex)
• Move DataStream and BinaryFormat constructors to DataStreamFactory (#111 by @pleonex)
• BinaryFormat does not create its own DataStream, it will dispose the stream passed in the constructor (#111 by @pleonex)
• Move BinaryFormat class to the Yarhl.IO namespace (0450b98 by @pleonex)
• Rename DataReader.ReadPadding to SkipPadding (#109 by @pleonex)
• Prevent changing length of a substream (#97 by @pleonex)
• Rename Replacer.Transform into TransformForward and TransformBackward (#101 by @pleonex)

Bugs

• Fix decoding issues in DataReader.ReadString (#137 by @pleonex)
• Fix writing large files (> 2GB) (#113 by @pleonex)
• Missing extensions defined by an executable (eed98a7 by @pleonex)
• Skip assemblies throwing BadImageFormatException (#125 by @Kaplas80)
• Changing substream length may be overwriting other stream sections (#97 by @pleonex)
• Fix converting from base types (#102 by @pleonex)
• Check if the return type after transforming implements IFormat (#103 by @pleonex)
• Do not dispose when changing to the same format instance (#119 by @pleonex)
• Documentation link and version improvements (#106 and #107 by @priverop)

Other changes

• Build and ship for .NET Framework 4.6.1 (TFM net461) for applications running in .NET Framework below 4.7.2 (d1588b9 by @pleonex)
• Run tests in latest frameworks: .NET Core 3.1, .NET Framework 4.8 and Mono 6 (#116 #137 by @pleonex)
• Consolidate CI in Azure DevOps and create test feed (#94 by @pleonex)
• Improvements in build system and project handling (#117 by @pleonex)
• Move SonarQube to SonarLint to get feedback while coding (#117 by @pleonex)
• Warnings cleanup (team work accross PR like #98 and #122)
• Change default branch to develop (b0ca82e by @pleonex)

Yarhl 2.0

New plugin discovery system and build system.

Special thanks to @priverop for the support, fixes and doc!

Infrastructure

• New plugin discovery system with MEF 2 to replace Mono.Addin.
• New build system with Cake. Used to build, test, validate and deploy code and documentation. It works in all OS and it's used by the CI too.
• Add on-line API documentation and overview
• Add contribution guidelines
• Deploy packages to NuGet
• Load plugin DLL from directory Plugins
• Target .NET Standard 2.0 for the project --> Tested in Mono, .NET Framework and .NET Core
• Fix warnings

Features and bug fixes

• Support BOM / Preamble in TextReader and TextWriter (#70)
• Support multilines in Po entry comments (#80)

Yarhl 1.0

First release with the new name Yahrl.

This is a rewrite of the old libgame library. It changes the way to work with Files (Node instead of GameFile), Format and conversions. It implements a new method of conversions, implementing new classes from IConverter<,> instead of implementing hard-coded methods in the Format class.

It also provides with converters for Po files and better ways to handle Streams and read / write binary and text files.

This release has been tested in many small programs to edit text, script and image files in game fan-translations.

0.4 - Initial Release

First Stable Release

It features a full library ready to work with ROM Hacking project. This version has been used in modime for the Ninokuni NDS Spanish Translation.