Windows compilation

[tclausen]

Hi,

We're trying to cross-compile this little program:

use yara::Compiler;

const RULES: &str = r#"
    rule contains_rust {
      strings:
        $rust = "rust" nocase
      condition:
        $rust
"#;

fn main() {
    let mut compiler = Compiler::new().unwrap();
    compiler.add_rules_str(RULES)
        .expect("Should have parsed rule");
    let rules = compiler.compile_rules()
        .expect("Should have compiled rules");
    let results = rules.scan_mem("I love Rust!".as_bytes(), 5)
        .expect("Should have scanned");
    assert!(results.iter().any(|r| r.identifier == "contains_rust"));
}

to Windows from Linux by issuing:

cargo build --bin thomas --target x86_64-pc-windows-gnu

[package]
name = "thomas"
version = "0.1.0"
authors = ["Sonny <[email protected]>"]
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[[bin]]
name = "thomas"
path = "src/mymain.rs"

[dependencies]
yara = "0.4.3"
yara-src = "0.1.2+3.11.0"

We have set the correct inc dir:
YARA_INCLUDE_DIR="../yara-3.11.0/libyara/include"

The error we get is:

warning: version requirement `0.1.2+3.11.0` for dependency `yara-src` includes semver metadata which will be ignored, removing the metadata is recommended to avoid confusion
   Compiling yara v0.4.3
error[E0308]: mismatched types
   --> /volumes/SoonrRepo/Users/tclausen/.cargo/registry/src/github.com-1ecc6299db9ec823/yara-0.4.3/src/internals/compiler.rs:115:13
    |
115 |             handle,
    |             ^^^^^^ expected `i32`, found *-ptr
    |
    = note:     expected type `i32`
            found raw pointer `*mut c_void`

error[E0308]: mismatched types
   --> /volumes/SoonrRepo/Users/tclausen/.cargo/registry/src/github.com-1ecc6299db9ec823/yara-0.4.3/src/internals/scan.rs:130:13
    |
130 |             handle,
    |             ^^^^^^ expected `i32`, found *-ptr
    |
    = note:     expected type `i32`
            found raw pointer `*mut c_void`

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0308`.
error: could not compile `yara`

To learn more, run the command again with --verbose.

[tclausen]

Hi Hugal31. Thanks for picking this up. We were kind of hoping this was us doing sth wrong... At our end yara-rust is part of a larger project we're trying to compile for Windows. Cross-compiling from Linux is so far the option we've had the most success with but we got stuck at the above point.

Can you recommend an alternative method for getting a running example working on Windows?

[Hugal31]

Hi,

I am currently investigating how to fix this bug. Unfortunately, I don't think there is a workaround until I find a fix.

[Hugal31]

Quick question: do you know if you or a dependency using yara-rust make use of the "bundled" feature of yara-rust ?

[tclausen]

We don't explicitly use "bundled" - we're not sure if any dependencies use it though.

[Hugal31]

I think I see the problem(s):

• You use yara-src in your dependencies. You should not use it directly. It is used by yara-sys if you use the vendored feature of yara-rust. It is not what is causing the issue here though.
• Cross-compiling doesn't work well with bindgen: there is no easy way to tell clang to use Windows headers when generating the bindings for yara-sys. Moreover, I don't know how the linking phase will work (if you statically link with a .a file, you will have problems).

I see two solutions:

• You can try to compile directly on Windows (it should work, as this pipeline says).
• I can try to make Windows cross-compilation work by adding a bundled binding file for Windows, and make sure you can link with a .DLL. However, I don't know when I can do that.

[tclausen]

Thanks a lot for looking into it and thanks for the link to the pipeline. We spend a lot of time compiling on Windows and ran into various problems with different compilers installed and diff header files. We'll try again with inspiration from that pipeline.

[roblabla]

Cross-compiling doesn't work well with bindgen: there is no easy way to tell clang to use Windows headers when generating the bindings for yara-sys. Moreover, I don't know how the linking phase will work (if you statically link with a .a file, you will have problems).

There should be no problem using bindgen with yara, assuming you have a correctly functioning cross-compilation toolchain (with appropriate environment variables to let it find the system headers). There is a pain point here: rust-lang/rust-bindgen#2009.

Here's my setup: I use https://github.com/roblabla/msvc-wine, which acquires the MSVC headers and libs, and sets up clang, lld and a few other tools to properly look for the MSVC headers. I then set the following environment variables:

# Used by cc-rs for linking
AR_x86_64_pc_windows_msvc="llvm-lib"
# Used by cc-rs for compiling
CC_x86_64_pc_windows_msvc = "/path/to/msvc-wine/bin/clang-cl"
# Tell bindgen where to find the msvc headers. SDKINCLUDE and MSVCDIR taken from the msvcenv.sh.
BINDGEN_EXTRA_CLANG_ARGS="--no-standard-includes -isystem ${MSVCDIR}/include -isystem ${SDKINCLUDE}/shared -isystem ${SDKINCLUDE}/ucrt -isystem ${SDKINCLUDE}/um -isystem ${SDKINCLUDE}/winrt"

After doing this, and using this PR for yara-src-rs, yara builds and links without issue, generating the bindings dynamically.

[Benji377]

I think I have a similar issue. On Windows, when trying to run the example, it tells me Clang is missing:

main.rs

use yara::Compiler;

const RULES: &str = r#"
    rule contains_rust {
      strings:
        $rust = "rust" nocase
      condition:
        $rust
    }
"#;

fn main() {
    let compiler = Compiler::new().unwrap();
    let compiler = compiler
        .add_rules_str(RULES)
        .expect("Should have parsed rule");
    let rules = compiler
        .compile_rules()
        .expect("Should have compiled rules");
    let results = rules
        .scan_mem("I love Rust!".as_bytes(), 5)
        .expect("Should have scanned");
    assert!(results.iter().any(|r| r.identifier == "contains_rust"));
}

Cargo.toml:

[package]
name = "experimenting"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
yara = "0.20.0"

Exception:

   Compiling yara-sys v0.20.0
error: failed to run custom build command for `yara-sys v0.20.0`

Caused by:
  process didn't exit successfully: `C:\Users\...\experimenting\target\debug\build\yara-sys-f32a719f96a9cdca\build-script-build` (exit code: 101)
  --- stdout
  cargo:rustc-link-lib=dylib=yara
  cargo:rerun-if-env-changed=YARA_LIBRARY_PATH
  cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64-pc-windows-msvc
  cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64_pc_windows_msvc

  --- stderr
  thread 'main' panicked at 'Unable to find libclang: "couldn't find any valid shared libraries matching: ['clang.dll', 'libclang.dll'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', C:\Users\...\.cargo\registry\src\index.crates.io-6f17d22bba15001f\bindgen-0.64.0\./lib.rs:2393:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I probably just need to download the missing dll?

[vthib]

yara uses bindgen by default which requires clang to be installed yes: https://rust-lang.github.io/rust-bindgen/requirements.html#clang

[Benji377]

Thank you, it might be useful to put this in the documentation for noobs like me that don't know it.

[HydraDragonAntivirus]

C:\Users\victim\Desktop\simpleantivirus>cargo build
Compiling yara-sys v0.24.0
error: failed to run custom build command for yara-sys v0.24.0

Caused by:
process didn't exit successfully: C:\Users\victim\Desktop\simpleantivirus\target\debug\build\yara-sys-8bfceb119aa58f74\build-script-build (exit code: 101)
--- stdout
cargo:rustc-link-lib=dylib=yara
cargo:rerun-if-env-changed=YARA_LIBRARY_PATH
cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64_pc_windows_msvc

--- stderr
wrapper.h:1:10: fatal error: 'yara.h' file not found
thread 'main' panicked at C:\Users\victim.cargo\registry\src\index.crates.io-6f17d22bba15001f\yara-sys-0.24.0\build.rs:442:43:
Unable to generate bindings: ClangDiagnostic("wrapper.h:1:10: fatal error: 'yara.h' file not found\n")
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace Still same even if I install clang and add them to system path.

[vthib]

@HydraDragonAntivirus You probably don't have yara installed and need to use the vendored feature. See #121 (comment)

[HydraDragonAntivirus]

I tried to compile from yara-sys folder then this happened. I probably making same mistake. : Compiling yara-sys v0.24.0 (C:\Users\victim\Downloads\yara-rust-master\yara-sys)
error: failed to run custom build command for yara-sys v0.24.0 (C:\Users\victim\Downloads\yara-rust-master\yara-sys)

Caused by:
process didn't exit successfully: C:\Users\victim\Downloads\yara-rust-master\target\debug\build\yara-sys-6df1c56e99ba77b3\build-script-build (exit code: 101)
--- stdout
cargo:rustc-link-lib=dylib=yara
cargo:rerun-if-env-changed=YARA_LIBRARY_PATH
cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64-pc-windows-msvc
cargo:rerun-if-env-changed=YARA_LIBRARY_PATH_x86_64_pc_windows_msvc

--- stderr
wrapper.h:1:10: fatal error: 'yara.h' file not found
thread 'main' panicked at yara-sys\build.rs:442:43:
Unable to generate bindings: ClangDiagnostic("wrapper.h:1:10: fatal error: 'yara.h' file not found\n")
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

[HydraDragonAntivirus]

Also can't install bindegen. But can install clang and already setted them.
#include "yara.h"

#if !defined (YR_MAJOR_VERSION) || !defined (YR_MINOR_VERSION)

error "Yara version macro not defined."

#endif

#if YR_MAJOR_VERSION != 4

if YR_MAJOR_VERSION < 4

error "Only Yara v4 is supported."

else

warning "Yara versons above v4 are not supported. Please use Yara v4."

endif

#endif and where is the yara.h?

[HydraDragonAntivirus]

I fixed the problem by set YARA_INCLUDE_DIR=C:\Users\victim\Desktop\hydradragonav adding yara.h and other yarac include things.