fix `+` in json #3663

shargon · 2025-01-08T19:08:24Z

Description

Fix #2612
Based on https://github.com/neo-project/neo/pull/2050/files

Type of change

Optimization (the change is only an optimization)
Style (the change is only a code style for better maintenance or standard purpose)
Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

JsonTest_Object

Test Configuration:

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged and published in downstream modules

src/Neo.Json/JToken.cs

roman-khimov · 2025-01-09T19:03:41Z

This can lead to some state diff, needs to be checked.

tests/Neo.UnitTests/SmartContract/UT_JsonSerializer.cs

cschuchardt88

This is how you do it Check out https://www.unicode.org/charts/PDF/U0000.pdf for Basic Latin Unicode block (U+0021-U+007F).

src/Neo.Json/JToken.cs

cschuchardt88

@shargon
I fixed the issue on the branch for you. Check and test changes.

src/Neo.Json/JToken.cs

cschuchardt88 · 2025-01-17T20:52:36Z

src/Neo.Json/JString.cs

@@ -76,7 +76,7 @@ public override T GetEnum<T>(bool ignoreCase = false)

        internal override void Write(Utf8JsonWriter writer)
        {
-            writer.WriteStringValue(Value);
+            writer.WriteRawValue($"\"{DefaultEncoder.Encode(Value).Replace("\\u002B", "+")}\"");


Keep in mind allowing the + (plus) sign is a security risk, reference: dotnet/runtime#35281
Only for XSS injection though.

Note: After searching for hours with no luck for solution. I came up with this solution on my own. Its dirty and could have bugs or exploits, however am 99% sure we don't. Need more tests.

But it works with newtonsoft (according your reference)

Newtonsoft uses it own custom library. It doesn't use dotnet built-in JSON libraries.

cschuchardt88 · 2025-01-17T20:53:27Z

tests/Neo.UnitTests/SmartContract/UT_JsonSerializer.cs


-            Assert.AreEqual(@"{""\uAAAA"":true}", parsed.ToString());
+            json = @" {""测试"":   true}";


Had to allow UnicodeRanges.CjkUnifiedIdeographs for these characters for json DefaultEncoder.

AnnaShaleva · 2025-01-20T16:18:23Z

tests/Neo.UnitTests/SmartContract/UT_JsonSerializer.cs

@@ -175,10 +175,13 @@ public void JsonTest_Object()

            Assert.AreEqual(@"{""test"":true}", parsed.ToString());

-            json = @" {""\uAAAA"":   true}";
+            json = @" {""test"":""+""} ";


Can we keep this test-case with \uAAAA? It's a nice compatibility test for NeoGo.

In fact I think this will be dropped, no way to do it without unsafe relaxing if we use native json

Doesnt use unsafe relaxing currently. And its working fine now.

Replace is a dirty fix, performance is affected always, not only when it contains +

RegEx.UnEscape? https://learn.microsoft.com/en-us/dotnet/api/system.text.regularexpressions.regex.unescape?view=net-9.0

Could use: https://github.com/dotnet/runtime/blob/4c224ab16d287c17dbc54236638db91266071465/src/libraries/System.Text.Json/src/System/Text/Json/Reader/JsonReaderHelper.Unescaping.cs#L69

AnnaShaleva · 2025-01-20T16:20:48Z

src/Neo.Json/JToken.cs

+
+        static JToken()
+        {
+            DefaultEncoder = JavaScriptEncoder.Create(UnicodeRanges.BasicLatin, UnicodeRanges.CjkUnifiedIdeographs);


Does it change the old behaviour except the + symbol?

Checkout #3663 (comment), but it should be the exactly the same with the addition of the + (plus sign).

Try fix

0917647

shargon commented Jan 8, 2025

View reviewed changes

src/Neo.Json/JToken.cs Outdated Show resolved Hide resolved

cschuchardt88 added Waiting for Review Work in Progress and removed Waiting for Review labels Jan 9, 2025

Merge branch 'master' into json-plus

6a77c33

Merge branch 'master' into json-plus

83b1394

cschuchardt88 reviewed Jan 13, 2025

View reviewed changes

tests/Neo.UnitTests/SmartContract/UT_JsonSerializer.cs Show resolved Hide resolved

cschuchardt88 reviewed Jan 15, 2025

View reviewed changes

src/Neo.Json/JToken.cs Outdated Show resolved Hide resolved

src/Neo.Json/JToken.cs Outdated Show resolved Hide resolved

Fixed unsafe relaxing

3c15c6f

cschuchardt88 reviewed Jan 17, 2025

View reviewed changes

Merge branch 'master' into json-plus

ebb00e7

cschuchardt88 reviewed Jan 17, 2025

View reviewed changes

src/Neo.Json/JToken.cs Outdated Show resolved Hide resolved

Allow plus sign +

9bb0fe7

cschuchardt88 reviewed Jan 17, 2025

View reviewed changes

cschuchardt88 assigned shargon and cschuchardt88 Jan 17, 2025

AnnaShaleva reviewed Jan 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix `+` in json #3663

fix `+` in json #3663

shargon commented Jan 8, 2025

roman-khimov commented Jan 9, 2025

cschuchardt88 left a comment •

edited

Loading

cschuchardt88 left a comment •

edited

Loading

cschuchardt88 Jan 17, 2025 •

edited

Loading

shargon Jan 17, 2025

cschuchardt88 Jan 18, 2025 •

edited

Loading

cschuchardt88 Jan 17, 2025

AnnaShaleva Jan 20, 2025

shargon Jan 20, 2025

cschuchardt88 Jan 20, 2025 •

edited

Loading

shargon Jan 20, 2025 •

edited

Loading

cschuchardt88 Jan 20, 2025 •

edited

Loading

AnnaShaleva Jan 20, 2025

cschuchardt88 Jan 20, 2025


		Assert.AreEqual(@"{""\uAAAA"":true}", parsed.ToString());
		json = @" {""测试"": true}";

fix + in json #3663

Are you sure you want to change the base?

fix + in json #3663

Conversation

shargon commented Jan 8, 2025

Description

Type of change

How Has This Been Tested?

Checklist:

roman-khimov commented Jan 9, 2025

cschuchardt88 left a comment • edited Loading

Choose a reason for hiding this comment

cschuchardt88 left a comment • edited Loading

Choose a reason for hiding this comment

cschuchardt88 Jan 17, 2025 • edited Loading

Choose a reason for hiding this comment

shargon Jan 17, 2025

Choose a reason for hiding this comment

cschuchardt88 Jan 18, 2025 • edited Loading

Choose a reason for hiding this comment

cschuchardt88 Jan 17, 2025

Choose a reason for hiding this comment

AnnaShaleva Jan 20, 2025

Choose a reason for hiding this comment

shargon Jan 20, 2025

Choose a reason for hiding this comment

cschuchardt88 Jan 20, 2025 • edited Loading

Choose a reason for hiding this comment

shargon Jan 20, 2025 • edited Loading

Choose a reason for hiding this comment

cschuchardt88 Jan 20, 2025 • edited Loading

Choose a reason for hiding this comment

AnnaShaleva Jan 20, 2025

Choose a reason for hiding this comment

cschuchardt88 Jan 20, 2025

Choose a reason for hiding this comment

fix `+` in json #3663

fix `+` in json #3663

cschuchardt88 left a comment •

edited

Loading

cschuchardt88 left a comment •

edited

Loading

cschuchardt88 Jan 17, 2025 •

edited

Loading

cschuchardt88 Jan 18, 2025 •

edited

Loading

cschuchardt88 Jan 20, 2025 •

edited

Loading

shargon Jan 20, 2025 •

edited

Loading

cschuchardt88 Jan 20, 2025 •

edited

Loading