Skip to content
This repository has been archived by the owner on Jan 15, 2025. It is now read-only.

(PA-6324) Add windowsfips-2016-x64 to agent-runtime-main and agent-runtime-7.x #900

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions configs/components/curl.rb
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
elsif platform.is_windows?
pkg.build_requires "runtime-#{settings[:runtime_project]}"
pkg.environment "PATH", "$(shell cygpath -u #{settings[:gcc_bindir]}):$(PATH)"
pkg.environment "NM" , "/usr/bin/nm" if platform.name =~ /windowsfips-2016/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what the implications are of using Cygwin nm with binaries built by mingw... I assume this problem would go away if we were using a newer version of mingw tool chain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was able to find 13.2.0 in the default repository(https://chocolatey.org/api/v2) of choco: https://community.chocolatey.org/packages/mingw
I think this is the latest version. It ships with nm 2.39. Unfortunately, it still results in the global_symbol_pipe variable being empty with this version.
The system nm (/usr/bin/nm) is version 2.42 which doesn't result the global_symbol_pipe issue. So, I manually set it to system nm.

I also tried with mingw 8.1.0 (nm version 2.25) which was available in the default repo, that also fails with the same error.

pkg.environment "CYGWIN", settings[:cygwin]
elsif platform.is_aix? && platform.name != 'aix-7.1-ppc'
pkg.environment "PKG_CONFIG_PATH", "/opt/puppetlabs/puppet/lib/pkgconfig"
Expand Down
2 changes: 1 addition & 1 deletion configs/components/openssl-1.0.2.rb
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@
'no-ssl3',
]

configure_flags += ['fips', "--with-fipsdir=#{settings[:prefix]}/usr/local/ssl/fips-2.0"] if platform.name =~ /windowsfips-2012r2/
configure_flags += ['fips', "--with-fipsdir=#{settings[:prefix]}/usr/local/ssl/fips-2.0"] if platform.name =~ /windowsfips-/

# Individual projects may provide their own openssl configure flags:
project_flags = settings[:openssl_extra_configure_flags] || []
Expand Down
16 changes: 15 additions & 1 deletion configs/components/ruby-2.7.8.rb
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,8 @@
'windows-2012r2-x64',
'windows-2012r2-x86',
'windows-2019-x64',
'windowsfips-2012r2-x64'
'windowsfips-2012r2-x64',
'windowsfips-2016-x64'
]

unless without_dtrace.include? platform.name
Expand Down Expand Up @@ -170,6 +171,19 @@
]
end

if(platform.name =~ /windowsfips-2016/)
# We need the below patch since during ruby build step for windowsfips-2016-x64 agent-runtime builds,
# the rbconfig.rb file that gets generated contains '\r' trailing character in 'ruby_version' config.
# We patch rbconfig.rb to remove the '\r' character.
# This patch has to run after the build step since rbconfig.rb is generated during the build step.
# This is sort of a hacky way to do this. We need to find why the '\r' character gets appended to
# 'ruby_version' field in the future so that this patch can be removed - PA-6902.
pkg.add_source("#{base}/rbconfig_win.patch")
pkg.build do
["TMP=/var/tmp /usr/bin/patch.exe --binary --strip=1 --fuzz=0 --ignore-whitespace --no-backup-if-mismatch < ../rbconfig_win.patch"]
joshcooper marked this conversation as resolved.
Show resolved Hide resolved
end
end

#########
# INSTALL
#########
Expand Down
16 changes: 15 additions & 1 deletion configs/components/ruby-3.2.5.rb
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,8 @@
'windows-2012r2-x64',
'windows-2012r2-x86',
'windows-2019-x64',
'windowsfips-2012r2-x64'
'windowsfips-2012r2-x64',
'windowsfips-2016-x64'
]

unless without_dtrace.include? platform.name
Expand Down Expand Up @@ -195,6 +196,19 @@
]
end

if(platform.name =~ /windowsfips-2016/)
# We need the below patch since during ruby build step for windowsfips-2016-x64 agent-runtime builds,
# the rbconfig.rb file that gets generated contains '\r' trailing character in 'ruby_version' config.
# We patch rbconfig.rb to remove the '\r' character.
# This patch has to run after the build step since rbconfig.rb is generated during the build step.
# This is sort of a hacky way to do this. We need to find why the '\r' character gets appended to
# 'ruby_version' field in the future so that this patch can be removed - PA-6902.
pkg.add_source("#{base}/rbconfig_win.patch")
pkg.build do
["TMP=/var/tmp /usr/bin/patch.exe --binary --strip=1 --fuzz=0 --ignore-whitespace --no-backup-if-mismatch < ../rbconfig_win.patch"]
end
shubhamshinde360 marked this conversation as resolved.
Show resolved Hide resolved
end

#########
# INSTALL
#########
Expand Down
54 changes: 54 additions & 0 deletions configs/platforms/windowsfips-2016-x64.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
platform "windowsfips-2016-x64" do |plat|
plat.vmpooler_template 'win-2016-fips-x86_64'

plat.servicetype 'windows'
visual_studio_version = '2017'
visual_studio_sdk_version = 'win8.1'

# We need to ensure we install chocolatey prior to adding any nuget repos. Otherwise, everything will fall over
plat.add_build_repository "https://artifactory.delivery.puppetlabs.net/artifactory/generic/buildsources/windows/chocolatey/install-chocolatey-1.4.0.ps1"
plat.provision_with "C:/ProgramData/chocolatey/bin/choco.exe feature enable -n useFipsCompliantChecksums"

plat.add_build_repository "https://artifactory.delivery.puppetlabs.net/artifactory/api/nuget/nuget"

# C:\tools is likely added by mingw, however because we also want to use that
# dir for vsdevcmd.bat we create it for safety
plat.provision_with "mkdir -p C:/tools"
# We don't want to install any packages from the chocolatey repo by accident
plat.provision_with "C:/ProgramData/chocolatey/bin/choco.exe sources remove -name chocolatey"

packages = [
"cmake",
"pl-gdbm-#{self._platform.architecture}",
"pl-iconv-#{self._platform.architecture}",
"pl-libffi-#{self._platform.architecture}",
"pl-pdcurses-#{self._platform.architecture}",
"pl-toolchain-#{self._platform.architecture}",
"pl-zlib-#{self._platform.architecture}",
"mingw-w64 -version 5.2.0 -debug",
]

packages.each do |name|
plat.provision_with("C:/ProgramData/chocolatey/bin/choco.exe install -y --no-progress #{name}")
end
# We use cache-location in the following install because msvc has several long paths
# if we do not update the cache location choco will fail because paths get too long
plat.provision_with "C:/ProgramData/chocolatey/bin/choco.exe install msvc.#{visual_studio_version}-#{visual_studio_sdk_version}.sdk.en-us -y --cache-location=\"C:\\msvc\" --no-progress"
# The following creates a batch file that will execute the vsdevcmd batch file located within visual studio.
# We create the following batch file under C:\tools\vsdevcmd.bat so we can avoid using both the %ProgramFiles(x86)%
# evironment var, as well as any spaces in the path when executing things with cygwin. This makes command execution
# through cygwin much easier.
#
# Note that the unruly \'s in the following string escape the following sequence to literal chars: "\" and then \""
plat.provision_with "touch C:/tools/vsdevcmd.bat && echo \"\\\"%ProgramFiles(x86)%\\Microsoft Visual Studio\\#{visual_studio_version}\\BuildTools\\Common7\\Tools\\vsdevcmd\\\"\" >> C:/tools/vsdevcmd.bat"

plat.install_build_dependencies_with "C:/ProgramData/chocolatey/bin/choco.exe install -y --no-progress"

plat.make "/usr/bin/make"
plat.patch "TMP=/var/tmp /usr/bin/patch.exe --binary"

plat.platform_triple "x86_64-w64-mingw32"

plat.package_type "archive"
plat.output_dir "windows"
end
2 changes: 1 addition & 1 deletion configs/projects/_shared-agent-components.rb
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
elsif platform.name =~ /^redhatfips-.*/
proj.component "openssl-1.1.1-fips"
else
proj.component "openssl-fips-2.0.16" if platform.name =~ /windowsfips-2012r2/ && proj.openssl_version =~ /1.0.2/
proj.component "openssl-fips-2.0.16" if platform.name =~ /windowsfips-/ && proj.openssl_version =~ /1.0.2/
proj.component "openssl-#{proj.openssl_version}"
end

Expand Down
4 changes: 2 additions & 2 deletions configs/projects/_shared-agent-settings.rb
Original file line number Diff line number Diff line change
Expand Up @@ -162,7 +162,7 @@
proj.setting(:openssl_version, '3.0')
elsif platform.name =~ /^redhatfips-/
proj.setting(:openssl_version, '1.1.1-fips')
elsif platform.name =~ /^windowsfips-2012r2/
elsif platform.name =~ /^windowsfips-/
proj.setting(:openssl_version, '1.0.2')
else
proj.setting(:openssl_version, '1.1.1')
Expand All @@ -183,7 +183,7 @@
proj.setting(:cflags, "#{proj.cppflags}")

ldflags = "-L#{proj.tools_root}/lib -L#{proj.gcc_root}/lib -L#{proj.libdir} -Wl,--nxcompat"
if platform.name !~ /windowsfips-2012r2/ || name != 'agent-runtime-7.x'
if platform.name !~ /windowsfips-/ || name != 'agent-runtime-7.x'
ldflags += ' -Wl,--dynamicbase'
end
proj.setting(:ldflags, ldflags)
Expand Down
22 changes: 22 additions & 0 deletions resources/patches/ruby_27/rbconfig_win.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
diff --git a/rbconfig.rb b/rbconfig.rb
index de4f173..87459fe 100644
--- a/rbconfig.rb
+++ b/rbconfig.rb
@@ -13,7 +13,7 @@ module RbConfig
raise "ruby lib version (2.7.8) doesn't match executable version (#{RUBY_VERSION})"

# Ruby installed directory.
- TOPDIR = File.dirname(__FILE__).chomp!("/lib/ruby/2.7.0\r/x64-mingw32")
+ TOPDIR = File.dirname(__FILE__).chomp!("/lib/ruby/2.7.0/x64-mingw32")
# DESTDIR on make install.
DESTDIR = TOPDIR && TOPDIR[/\A[a-z]:/i] || '' unless defined? DESTDIR
# The hash configurations stored.
@@ -54,7 +54,7 @@ module RbConfig
CONFIG["sitedir"] = "$(rubylibprefix)/site_ruby"
CONFIG["rubyarchdir"] = "$(rubylibdir)/$(arch)"
CONFIG["rubylibdir"] = "$(rubylibprefix)/$(ruby_version)"
- CONFIG["ruby_version"] = "2.7.0\r"
+ CONFIG["ruby_version"] = "2.7.0"
Copy link
Contributor

@joshcooper joshcooper Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe ruby_version is detected during the ruby build by https://github.com/ruby/ruby/blob/31d0f1a2e7dbfb60731d1f05b868e1d578cda493/configure.ac#L4203 It emits a conftest.c program, compiles and runs it, and uses sed to extract the version. As mentioned in https://cygwin.com/cygwin-ug-net/using-textbinary.html, cygwin sed "Most other programs (such as sed, cmp, tr) use the default mode." The default behavior depends on the filesystem that is mounted. I'm guessing the win2016 cygwin is configured differently than 2012r2. Is it possible 2016 has mounted / as text? See https://cygwin.com/cygwin-ug-net/mount.html for details

CONFIG["sitearch"] = "x64-msvcrt"
CONFIG["arch"] = "x64-mingw32"
CONFIG["sitearchincludedir"] = "$(includedir)/$(sitearch)"
22 changes: 22 additions & 0 deletions resources/patches/ruby_32/rbconfig_win.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
diff --git a/rbconfig.rb b/rbconfig.rb
index 592dd52..a994b0a 100644
--- a/rbconfig.rb
+++ b/rbconfig.rb
@@ -13,7 +13,7 @@ module RbConfig
raise "ruby lib version (3.2.5) doesn't match executable version (#{RUBY_VERSION})"

# Ruby installed directory.
- TOPDIR = File.dirname(__FILE__).chomp!("/lib/ruby/3.2.0\r/x64-mingw32")
+ TOPDIR = File.dirname(__FILE__).chomp!("/lib/ruby/3.2.0/x64-mingw32")
# DESTDIR on make install.
DESTDIR = TOPDIR && TOPDIR[/\A[a-z]:/i] || '' unless defined? DESTDIR
# The hash configurations stored.
@@ -55,7 +55,7 @@ module RbConfig
CONFIG["sitedir"] = "$(rubylibprefix)/site_ruby"
CONFIG["rubyarchdir"] = "$(rubylibdir)/$(arch)"
CONFIG["rubylibdir"] = "$(rubylibprefix)/$(ruby_version)"
- CONFIG["ruby_version"] = "3.2.0\r"
+ CONFIG["ruby_version"] = "3.2.0"
CONFIG["sitearch"] = "x64-msvcrt"
CONFIG["arch"] = "x64-mingw32"
CONFIG["sitearchincludedir"] = "$(includedir)/$(sitearch)"
Loading