Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Driver quantize fp8 update #3715

Merged
merged 41 commits into from
Jan 22, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
70336db
Initial
CharlieL7 Oct 3, 2024
b41c8b6
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into ocp_to_fnuz
CharlieL7 Nov 12, 2024
bdebeb5
progress
CharlieL7 Nov 15, 2024
a1fb21e
cleanup
CharlieL7 Nov 22, 2024
b8e2041
remove unneeded files
CharlieL7 Nov 22, 2024
8366434
Fix bit_cast kernel
CharlieL7 Nov 22, 2024
a15e5a4
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into bit_cast_op
CharlieL7 Nov 22, 2024
be5d9a0
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into bit_cast_op
CharlieL7 Nov 22, 2024
3e08ab2
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into bit_cast_op
CharlieL7 Nov 25, 2024
7b40796
Merge branch 'bit_cast_op' of github.com:ROCm/AMDMIGraphX into bit_ca…
CharlieL7 Nov 25, 2024
697d459
progress
CharlieL7 Nov 27, 2024
4b6c8c1
fix template for gpu bit_cast
CharlieL7 Nov 27, 2024
531150f
Merge branch 'develop' into bit_cast_op
CharlieL7 Nov 27, 2024
d53ac35
Merge branch 'bit_cast_op' of github.com:ROCm/AMDMIGraphX into ocp_to…
CharlieL7 Nov 27, 2024
95a3cd7
first implementation
CharlieL7 Nov 27, 2024
98d8760
progress
CharlieL7 Dec 4, 2024
7357367
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into ocp_to_fnuz
CharlieL7 Dec 4, 2024
e3d84fc
Fixes and first test works
CharlieL7 Dec 4, 2024
dac07c2
formatting
CharlieL7 Dec 4, 2024
06b94b8
Added ref tests
CharlieL7 Dec 5, 2024
3e5d3a8
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into ocp_to_fnuz
CharlieL7 Dec 9, 2024
df0202e
Cleanup
CharlieL7 Dec 10, 2024
0318f32
initial
CharlieL7 Dec 10, 2024
3b48242
temporary
CharlieL7 Dec 10, 2024
b373d10
disable simpilify_qdq in quantization_8bits
CharlieL7 Dec 10, 2024
28aab5f
revert
CharlieL7 Dec 10, 2024
7e0142f
disable extra passes after quantize_8bits
CharlieL7 Dec 10, 2024
0a4d6bf
add verify test
CharlieL7 Dec 11, 2024
c94c520
Fix bug with __builtin_nan(string)
CharlieL7 Dec 13, 2024
d025e47
Merge branch 'develop' into ocp_to_fnuz
CharlieL7 Dec 13, 2024
0cddfbf
separate quantizable ops
CharlieL7 Dec 13, 2024
3c36b9b
Merge branch 'ocp_to_fnuz' of github.com:ROCmSoftwarePlatform/AMDMIGr…
CharlieL7 Dec 13, 2024
b62a304
Merge branch 'ocp_to_fnuz' of github.com:ROCm/AMDMIGraphX into driver…
CharlieL7 Dec 13, 2024
302adac
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into driver_qua…
CharlieL7 Jan 9, 2025
9ba6fe2
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into driver_qua…
CharlieL7 Jan 14, 2025
1de1d22
Merge branch 'develop' into driver_quantize_fp8_update
TedThemistokleous Jan 15, 2025
dac22d0
Merge branch 'develop' into driver_quantize_fp8_update
causten Jan 15, 2025
083a9da
Fix the gpu context test
CharlieL7 Jan 20, 2025
39592ae
Merge branch 'driver_quantize_fp8_update' of github.com:ROCm/AMDMIGra…
CharlieL7 Jan 20, 2025
1ddf7a1
Merge branch 'develop' of github.com:ROCm/AMDMIGraphX into driver_qua…
CharlieL7 Jan 20, 2025
9fc2e97
licensing update
CharlieL7 Jan 21, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 3 additions & 22 deletions src/quantization.cpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/*
* The MIT License (MIT)
*
* Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved.
* Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
Expand Down Expand Up @@ -168,10 +168,7 @@ void quantize_8bits(program& prog,
}

run_passes(prog,
{quantize_8bits_pass{precision, *quant_8bit_params},
simplify_qdq{},
optimize_module{},
dead_code_elimination{}},
{quantize_8bits_pass{precision, *quant_8bit_params}, dead_code_elimination{}},
quant_tracer());
}

Expand Down Expand Up @@ -208,23 +205,7 @@ void quantize_fp8(program& prog, const target& t, const std::vector<parameter_ma
supported_ins_names.insert(ins->name());
}
}
auto gfx_has_fp8fnuz = [&]() {
if(t.name() == "gpu")
{
auto context_value = t.get_context().to_value();
auto device_name = context_value["gfx_name"].to<std::string>();
return (starts_with(device_name, "gfx9") and device_name >= "gfx940");
}
return false;
};
if(gfx_has_fp8fnuz())
{
quantize_8bits(prog, t, shape::fp8e4m3fnuz_type, calibration, supported_ins_names);
}
else
{
quantize_8bits(prog, t, shape::fp8e4m3fn_type, calibration, supported_ins_names);
}
quantize_8bits(prog, t, shape::fp8e4m3fn_type, calibration, supported_ins_names);
}
} // namespace MIGRAPHX_INLINE_NS
} // namespace migraphx
3 changes: 1 addition & 2 deletions src/targets/gpu/include/migraphx/gpu/context.hpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/*
* The MIT License (MIT)
*
* Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved.
* Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
Expand Down Expand Up @@ -311,7 +311,6 @@ struct context
value result;
result["events"] = events.size();
result["streams"] = current_device->nstreams();
result["gfx_name"] = get_current_device().get_gfx_name();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this removed on serialization?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this earlier when getting FP8 OCP in to query the gfx number from the driver. We could keep it, but it would not be used anywhere anymore.


return result;
}
Expand Down
7 changes: 2 additions & 5 deletions test/gpu/context_serialize.cpp
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/*
* The MIT License (MIT)
*
* Copyright (c) 2015-2024 Advanced Micro Devices, Inc. All rights reserved.
* Copyright (c) 2015-2025 Advanced Micro Devices, Inc. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
Expand Down Expand Up @@ -33,17 +33,14 @@ TEST_CASE(gpu_context_serialize)
migraphx::context ctx = migraphx::gpu::context{0, 3};

auto v = ctx.to_value();
EXPECT(v.size() == 3);
EXPECT(v.size() == 2);

EXPECT(v.contains("events"));
EXPECT(v.at("events").without_key().to<std::size_t>() == 0);

EXPECT(v.contains("streams"));
EXPECT(v.at("streams").without_key().to<std::size_t>() == 3);

EXPECT(v.contains("gfx_name"));
EXPECT(not v.at("gfx_name").without_key().to<std::string>().empty());

migraphx::gpu::context g_ctx;
g_ctx.from_value(v);

Expand Down
Loading