Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor test apps to use unit-test framework #4014

Merged
merged 96 commits into from
Jan 8, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
96 commits
Select commit Hold shift + click to select a range
cdd4380
Initial work on unittest framework, tested
bennylp Jun 8, 2024
eaef323
Finished reorganizing pjlib-test to use unit-test framework
bennylp Jun 11, 2024
93f2848
Fix big problem where performance improvement is not observed with th…
bennylp Jun 12, 2024
51d6d93
Tidying up global vars in pjlib-test. Add test options: stop on error…
bennylp Jun 13, 2024
9383baa
Change the PJ_TEST_XX() signature to be more generic and does not for…
bennylp Jun 13, 2024
1d842db
Modifications to some existing tests to use unit-test test macros
bennylp Jun 13, 2024
d8bd130
Updated VS projects with argparse.h and unittest.h
bennylp Jun 14, 2024
c8f93dd
Fix warnings on Windows
bennylp Jun 19, 2024
56d221a
Disable parallel unit-testing for ioqueue stress test on Windows beca…
bennylp Jun 19, 2024
cbefbdf
Non-parallel test case will now run exclusively (previously it did no…
bennylp Jun 20, 2024
19f9f17
Add pjlib-test/test_util.h for common utilities for parsing, configur…
bennylp Jun 20, 2024
b2e3108
Dirty hack to fix error message being displayed when user selected es…
bennylp Jun 20, 2024
13db0c6
Replace PJ_TEST_PARALLEL with PJ_TEST_EXCLUSIVE
bennylp Jun 21, 2024
6bfb5d2
Ported pjnath-test to use unit-testing framework, with limited speed-…
bennylp Jun 21, 2024
2b735d0
Large modifications in pjnath-test to speed-up test. It is fast now (…
bennylp Jun 21, 2024
b8594fe
Ported pjmedia-test to use unit-test
bennylp Jun 22, 2024
af408ea
Porting of pjsip test to unit testing framework. Not much of speed im…
bennylp Jun 25, 2024
4f5d62c
Merged changes from master
bennylp Jun 26, 2024
2e5fa0e
Modifications in pj_argparse API (changed get() to get_bool() and add…
bennylp Jun 26, 2024
d4b267e
Refactor tsx_uas_test() to allow parallel testing
bennylp Jun 26, 2024
f7f7392
Parallelize tsx_uac_test()
bennylp Jun 27, 2024
64d5a61
Further effort to parallize tests in pjsip-test. Discoverd major issu…
bennylp Jun 27, 2024
2b90407
Showing PJLIB config is optional with cmd line option
bennylp Jun 30, 2024
9caadb2
Bug fixing failed pjsip tests when running in parallel mode
bennylp Jun 30, 2024
e886488
Finished paralleizing all pjsip tests except one
bennylp Jul 1, 2024
47d94bb
Continuing correcting errors
bennylp Jul 2, 2024
ed3ac46
Add test shuffle feature
bennylp Jul 2, 2024
97b99c1
Merge #4006 and fix transport_loop_test error
bennylp Jul 2, 2024
46ef3e6
Modify CI workflows to use standard arguments: -w 3 --shuffle. These …
bennylp Jul 3, 2024
b012957
Updated due to change in argparse API signature (swap arg order)
bennylp Jul 10, 2024
37272e8
Merged changes from master (unit test, argparse, etc.)
bennylp Jul 10, 2024
7117775
Removed unittest.md (it was draft for the PR)
bennylp Jul 11, 2024
cfe3a3a
Fix the use of GitHub repository vars
bennylp Jul 11, 2024
3682785
Merge branch 'master' into unittest-framework
bennylp Jul 12, 2024
e1b1e45
Tidying up minor conflicting merge in VC projects and pjlib.h
bennylp Jul 12, 2024
0246ac9
Attempt to fix assertion failure when tdata is being accessed after r…
bennylp Jul 13, 2024
d6b892f
Fixing failure in transport_loop_test(), first in loop_resolve_error(…
bennylp Jul 13, 2024
bd96226
Attempt to fix unresolved winmm.lib API (e.g. timeKillEvent, timeGetT…
bennylp Jul 16, 2024
c930d4c
Split exclusive part of transport_loop_test
bennylp Jul 16, 2024
bc8f174
CI mods: split steps, envs, shorten job names
bennylp Jul 17, 2024
49b3b5c
Fix failed tsx_basic_test
bennylp Jul 17, 2024
5148484
Restore previously shortened job names
bennylp Jul 18, 2024
cf5d2cc
Replace assertion with PJ_TEST_XX in resolver_test
bennylp Jul 18, 2024
4cfff2f
Add --stdout-buf and --stderr-buf options in test apps to control std…
bennylp Jul 18, 2024
72d425e
Fix missing sipp exe on Windows CI
bennylp Jul 18, 2024
311edb9
Fix wrong CI args on Mac. Use -j for faster make
bennylp Jul 18, 2024
78c9adf
Replace pj_rand() with own rand in unittest because rand() yields dif…
bennylp Jul 19, 2024
b08343b
Fix swig make error on Linux and runall.py error reading log file on …
bennylp Jul 19, 2024
7b86a22
Attempt to fix test repeatability by 1) delete all calls to pj_srand(…
bennylp Jul 19, 2024
7dc7ee9
Merge branch 'master' into unittest-framework
bennylp Dec 17, 2024
d0a68f2
Fixed inexistant function
bennylp Dec 17, 2024
988f65d
Relaxing the strictness of the test since sometimes it raises error
bennylp Dec 17, 2024
6e9e4b2
Set regc_test() exclusive because it crashes sometimes, probably conc…
bennylp Dec 17, 2024
6dbb1ba
Modified thread counter to unsigned long (from pj_uint32_t) since the…
bennylp Dec 18, 2024
0f3941c
Fixed port double destruction in mips_test() and include benchmark te…
bennylp Dec 18, 2024
b2a5134
Use any port since sometimes test fails with address in use error
bennylp Dec 18, 2024
420595f
Merge branch 'master' into unittest-framework
bennylp Dec 18, 2024
aea2049
Merge branch 'mips_test' into unittest-framework
bennylp Dec 18, 2024
b69ac09
Merge branch 'master' into unittest-framework
bennylp Dec 18, 2024
d8f0280
Fix conflicted return value in udp_ioqueue_test() and let it immediat…
bennylp Dec 18, 2024
cb5f7f8
Restore sleep(0) in thread test since without it the test may occasio…
bennylp Dec 18, 2024
6d185a6
Various attempt to fix fluke error in resolve_test.c: 1) servers use …
bennylp Dec 19, 2024
01712f1
Merge branch 'master' into unittest-framework
bennylp Dec 19, 2024
97b0949
More relaxed packet count tests in resolver_test
bennylp Dec 19, 2024
d283d3b
Use any port instead of hardcoded one in udp ioqueue unregister_test …
bennylp Dec 19, 2024
cb2074d
Merge branch 'master' into unittest-framework
bennylp Dec 19, 2024
734312f
Rollback previous changes in resolver_test that relaxed packet count …
bennylp Dec 20, 2024
3203ce8
Protect access to pool from worker thread with mutex in resolver_test
bennylp Dec 20, 2024
aea63e2
Faster resolver_test time by reducing timeout
bennylp Dec 20, 2024
1506d37
Merge branch 'unittest-framework' of https://github.com/pjsip/pjproje…
bennylp Dec 20, 2024
58602a4
Use high number port to make it less prone to bind error
bennylp Dec 20, 2024
dadf081
Remove hardcoded port number, replace with bind to any
bennylp Dec 20, 2024
0cb6caf
Merge branch 'master' of https://github.com/pjsip/pjproject into unit…
bennylp Dec 25, 2024
c9c385d
Fix SSL to continue decrypting data after renego completes
nanangizz Dec 25, 2024
16b4a7a
Fixed race condition when registering SIP module in transport test
bennylp Dec 25, 2024
03b1c59
Merge branch 'ssl-sock-renego' into unittest-framework
bennylp Dec 25, 2024
eb87000
Merge branch 'master' into unittest-framework
bennylp Dec 31, 2024
c5abaae
Merge branch 'unittest-framework' of https://github.com/pjsip/pjproje…
bennylp Dec 31, 2024
abfbf55
Disable loading TLS cert in TURN sock test if SChannel is used
bennylp Dec 31, 2024
87beeb4
Add (missing!) pjnath-test in Windows CI
bennylp Dec 31, 2024
879fd54
Fix syntax error in ci-win.yml and renamed lin->ubuntu in ci-linux
bennylp Dec 31, 2024
56b6ff7
Merge branch 'unittest-framework' of https://github.com/pjsip/pjproje…
bennylp Dec 31, 2024
17c94d4
Temporary workaround for MacOS rwmutex deadlock issue
bennylp Jan 1, 2025
6b655e3
More jobs/tests in CI Mac, and changed names
bennylp Jan 1, 2025
b9c5470
Fix silly typo in config_site_test.h
bennylp Jan 1, 2025
63c92d1
Prettyfy CI job names, added more test steps
bennylp Jan 1, 2025
ee3ca62
Fix CI: ffmpeg lib path, faster git clone
bennylp Jan 1, 2025
4d9e650
CI: simplify job names, add some audio checking
bennylp Jan 1, 2025
45f3b23
CI: Disable SDL check because vid renderer is not available on CI mac…
bennylp Jan 1, 2025
86be063
Fixed missing ffmpeg shared lib when running pjmedia test
bennylp Jan 1, 2025
8abba1c
CI Mac: install gnutls
sauwming Jan 3, 2025
24852bb
Lookup renderer in video codec test
sauwming Jan 3, 2025
090a842
CI MacOS: attempt to fix failed OpenSSL test
bennylp Jan 3, 2025
ef918f8
Merge branch 'unittest-framework' of https://github.com/pjsip/pjproje…
bennylp Jan 3, 2025
25b806a
CI Mac: better test split to make duration more uniform across jobs
bennylp Jan 4, 2025
22b7038
Replace deprecated egrep with grep -E
bennylp Jan 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
191 changes: 138 additions & 53 deletions .github/workflows/ci-linux.yml
Original file line number Diff line number Diff line change
@@ -1,25 +1,34 @@
name: CI Linux
name: CI Ubuntu
on:
push:
branches:
- 'master'
pull_request:
types: [opened, synchronize, reopened]
env:
CI_ARGS: ${{ vars.CI_UBUNTU_ARGS }}
CI_MODE: ${{ vars.CI_MODE }}
MAKE_FAST: make -j 3
jobs:
build-ubuntu-default:
# checking pure lib source distribution with plain configure & make
default-build:
# checking pure lib source distribution with plain configure & make
runs-on: ubuntu-latest
name: Default / build only
steps:
- uses: actions/checkout@v2
- name: configure
run: ./configure
- name: make
run: make
run: $MAKE_FAST
- name: get SSL info
run: pjlib/bin/pjlib-test-`make infotarget` --config --list | grep SSL
- name: verify oepnssl is used
run: pjlib/bin/pjlib-test-`make infotarget` --config --list | grep -E 'PJ_SSL_SOCK_IMP\s+:\s+1'

ubuntu-default-full-bundle-1:
# full bundle: enable all codecs + AEC + DTLS
# full bundle 1: running pjlib, pjlib-util, pjmedia, and pjsua tests
default-full-bundle-1:
# full bundle: enable all codecs + AEC + DTLS
runs-on: ubuntu-latest
name: Default / pjmedia,pjsua
steps:
- uses: actions/checkout@v2
- name: install dependencies
Expand All @@ -29,19 +38,33 @@ jobs:
- name: configure
run: CFLAGS="-g -fPIC" CXXFLAGS="-g -fPIC" LDFLAGS="-rdynamic" ./configure
- name: make
run: make
run: $MAKE_FAST
- name: swig bindings
run: cd pjsip-apps/src/swig && make
- name: set up Python 3.10 for pjsua test
uses: actions/setup-python@v2
with:
python-version: '3.10'
- name: unit tests
run: make pjlib-test-ci pjlib-util-test pjmedia-test pjsua-test
- name: capture pjsua capabilities
run: |
cat << EOF | pjsip-apps/bin/pjsua-`make infotarget` --log-level 3 > pjsua-caps
Cp
xx
vid dev list
vid codec list
q
EOF
cat pjsua-caps
- name: ensure AMR codec is installed
run: cat pjsua-caps | grep -E 'AMR/8000'
bennylp marked this conversation as resolved.
Show resolved Hide resolved
- name: pjmedia-test
run: make pjmedia-test
- name: pjsua-test
run: make pjsua-test

ubuntu-default-full-bundle-2:
# full bundle 2: running pjnath test
default-full-bundle-2:
runs-on: ubuntu-latest
name: Default / pjlib,util,pjnath
steps:
- uses: actions/checkout@v2
- name: install dependencies
Expand All @@ -51,13 +74,17 @@ jobs:
- name: configure
run: CFLAGS="-g" LDFLAGS="-rdynamic" ./configure
- name: make
run: make
- name: unit tests
run: $MAKE_FAST
- name: pjlib-test
run: make pjlib-test
- name: pjlib-util-test
run: make pjlib-util-test
- name: pjnath-test
run: make pjnath-test

ubuntu-default-full-bundle-3:
# full bundle 3: running pjsip test
default-full-bundle-3:
runs-on: ubuntu-latest
name: Default / pjsip
steps:
- uses: actions/checkout@v2
- name: install dependencies
Expand All @@ -67,126 +94,184 @@ jobs:
- name: configure
run: CFLAGS="-g" LDFLAGS="-rdynamic" ./configure
- name: make
run: make
- name: unit tests
run: $MAKE_FAST
- name: pjsip-test
run: make pjsip-test

build-ubuntu-no-tls:
# no TLS
no-tls:
runs-on: ubuntu-latest
name: No SSL / pjlib,pjsip
steps:
- uses: actions/checkout@v2
- name: install dependencies
run: sudo apt-get install -y swig
- name: configure
run: CFLAGS="-fPIC" CXXFLAGS="-fPIC" ./configure --disable-ssl
- name: make
run: make
run: $MAKE_FAST
- name: swig bindings
run: cd pjsip-apps/src/swig && make
- name: pjlib-test
run: make pjlib-test
- name: pjsip-test
run: make pjsip-test

# build-ubuntu-openssl
# TLS: with OpenSSL (same as build-ubuntu-default)

build-ubuntu-gnu-tls:
# TLS: with GnuTLS
gnu-tls:
runs-on: ubuntu-latest
name: GnuTLS / pjlib,pjnath,pjsip
steps:
- uses: actions/checkout@v2
- name: install dependencies
run: sudo apt-get update && sudo apt-get install -y --fix-missing swig libgnutls28-dev
- name: configure
run: CFLAGS="-fPIC" CXXFLAGS="-fPIC" ./configure --with-gnutls=/usr/
- name: make
run: make
run: $MAKE_FAST
- name: swig bindings
run: cd pjsip-apps/src/swig && make
- name: get SSL info
run: pjlib/bin/pjlib-test-`make infotarget` --config --list | grep SSL
- name: verify gnu tls is used
run: pjlib/bin/pjlib-test-`make infotarget` --config --list | grep -E 'PJ_SSL_SOCK_IMP\s+:\s+2'
- name: pjlib-test
run: make pjlib-test
- name: pjnath-test
run: make pjnath-test
- name: pjsip-test
run: make pjsip-test

ubuntu-video-openh264-1:
# video: video enabled with vpx and openh264
# video 1: running pjlib, pjlib-util, pjmedia, and pjsua tests
vid-openh264-1:
runs-on: ubuntu-latest
name: OpenH264+VPX / pjmedia,pjsua
steps:
- uses: actions/checkout@v2
- name: install dependencies
run: sudo apt-get install -y swig nasm sip-tester libvpx-dev libopencore-amrnb-dev
run: sudo apt-get install -y swig nasm sip-tester libvpx-dev libopencore-amrnb-dev libsdl2-dev
- name: get openh264
run: git clone --single-branch --branch openh264v2.1.0 https://github.com/cisco/openh264.git
run: git clone --depth 1 --single-branch --branch openh264v2.1.0 https://github.com/cisco/openh264.git
- name: build openh264
run: cd openh264 && make && sudo make install && sudo ldconfig
run: cd openh264 && $MAKE_FAST && sudo make install && sudo ldconfig
- name: config site
run: cd pjlib/include/pj && cp config_site_test.h config_site.h && echo "#define PJMEDIA_HAS_VIDEO 1" >> config_site.h
- name: configure
run: CFLAGS="-g -fPIC -DHAS_VID_CODEC_TEST=0" CXXFLAGS="-g -fPIC" LDFLAGS="-rdynamic" ./configure
- name: make
run: make
run: $MAKE_FAST
- name: swig bindings
run: cd pjsip-apps/src/swig && make
- name: set up Python 3.10 for pjsua test
uses: actions/setup-python@v4
with:
python-version: '3.10'
- name: unit tests
run: make pjlib-test-ci pjlib-util-test pjmedia-test pjsua-test
- name: capture pjsua capabilities
run: |
cat << EOF | pjsip-apps/bin/pjsua-`make infotarget` --log-level 3 > pjsua-caps
Cp
xx
vid dev list
vid codec list
q
EOF
cat pjsua-caps
- name: ensure AMR codec is installed
run: cat pjsua-caps | grep -E 'AMR/8000'
- name: ensure H264 codec is installed
run: cat pjsua-caps | grep -E 'H264/'
- name: ensure VP8 codec is installed
run: cat pjsua-caps | grep -E 'VP8/'
# SDL error: no available vid dev
#- name: ensure SDL is installed
# run: cat pjsua-caps | grep -E '\[SDL\]\[render\]'
- name: pjmedia-test
run: make pjmedia-test
- name: pjsua-test
run: make pjsua-test

ubuntu-video-openh264-2:
# video 2: running pjnath test
vid-openh264-2:
runs-on: ubuntu-latest
name: OpenH264+VPX / pjlib,util,pjnath
steps:
- uses: actions/checkout@v2
- name: install dependencies
run: sudo apt-get install -y nasm libvpx-dev libopencore-amrnb-dev
run: sudo apt-get install -y nasm libvpx-dev libopencore-amrnb-dev libsdl2-dev
- name: get openh264
run: git clone --single-branch --branch openh264v2.1.0 https://github.com/cisco/openh264.git
run: git clone --depth 1 --single-branch --branch openh264v2.1.0 https://github.com/cisco/openh264.git
- name: build openh264
run: cd openh264 && make && sudo make install && sudo ldconfig
run: cd openh264 && $MAKE_FAST && sudo make install && sudo ldconfig
- name: config site
run: cd pjlib/include/pj && cp config_site_test.h config_site.h && echo "#define PJMEDIA_HAS_VIDEO 1" >> config_site.h
- name: configure
run: CFLAGS="-g" LDFLAGS="-rdynamic" ./configure
- name: make
run: make
- name: unit tests
run: $MAKE_FAST
- name: pjlib-test
run: make pjlib-test
- name: pjlib-util-test
run: make pjlib-util-test
- name: pjnath-test
run: make pjnath-test

ubuntu-video-openh264-3:
# video: 3: running pjsip test
vid-openh264-3:
runs-on: ubuntu-latest
name: OpenH264+VPX / pjsip
steps:
- uses: actions/checkout@v2
- name: install dependencies
run: sudo apt-get install -y nasm libvpx-dev libopencore-amrnb-dev
run: sudo apt-get install -y nasm libvpx-dev libopencore-amrnb-dev libsdl2-dev
- name: get openh264
run: git clone --single-branch --branch openh264v2.1.0 https://github.com/cisco/openh264.git
run: git clone --depth 1 --single-branch --branch openh264v2.1.0 https://github.com/cisco/openh264.git
- name: build openh264
run: cd openh264 && make && sudo make install && sudo ldconfig
run: cd openh264 && $MAKE_FAST && sudo make install && sudo ldconfig
- name: config site
run: cd pjlib/include/pj && cp config_site_test.h config_site.h && echo "#define PJMEDIA_HAS_VIDEO 1" >> config_site.h
- name: configure
run: CFLAGS="-g" LDFLAGS="-rdynamic" ./configure
- name: make
run: make
- name: unit tests
run: $MAKE_FAST
- name: pjsip-test
run: make pjsip-test

build-ubuntu-video-ffmpeg:
# video enabled with vpx and ffmpeg and x264
vid-ffmpeg:
runs-on: ubuntu-latest
name: FFMPEG+x264 / pjmedia
steps:
- uses: actions/checkout@v2
- name: install dependencies
run: sudo apt-get install -y swig nasm libx264-dev libvpx-dev
run: sudo apt-get install -y swig nasm libx264-dev libvpx-dev libsdl2-dev
- name: get ffmpeg
run: git clone --single-branch --branch release/4.2 https://github.com/FFmpeg/FFmpeg.git
run: git clone --depth 1 --single-branch --branch release/4.2 https://github.com/FFmpeg/FFmpeg.git
- name: configure ffmpeg
run: cd FFmpeg && ./configure --enable-shared --disable-static --enable-gpl --enable-libx264
- name: build ffmpeg
run: cd FFmpeg && make -j10 && sudo make install
run: cd FFmpeg && $MAKE_FAST && sudo make install
- name: config site
run: echo -e "#define PJMEDIA_HAS_VIDEO 1\n" > pjlib/include/pj/config_site.h
- name: configure
run: CFLAGS="-fPIC" CXXFLAGS="-fPIC" ./configure
- name: make
run: make
run: $MAKE_FAST
- name: swig bindings
run: cd pjsip-apps/src/swig && make
- name: capture pjsua capabilities
run: |
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
cat << EOF | pjsip-apps/bin/pjsua-`make infotarget` --log-level 3 > pjsua-caps
Cp
xx
vid dev list
vid codec list
q
EOF
cat pjsua-caps
- name: ensure H264 codec is installed
run: cat pjsua-caps | grep -E 'H264/'
- name: ensure VP8 codec is installed
run: cat pjsua-caps | grep -E 'VP8/'
# SDL error: no available vid dev
#- name: ensure SDL is installed
# run: cat pjsua-caps | grep -E '\[SDL\]\[render\]'
- name: pjmedia-test
run: export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH && make pjmedia-test
Loading
Loading