Adding CI to your ParaView plugin and VTK modules

CI: Continuous Integration
MR: Merge Request
PR: Pull Request

In the VTK/ParaView ecosystem, there are a number of projects that rely on VTK and ParaView as dependency and are not part of VTK/ParaView per se, eg Flow3D, Salome, Themys, F3D, PVGeo, vtk-dicom, pyvista, vedo, meshio, slicer, CTK, cmb, ttk.

It has always been complex for these kind of project to be able to build and run test easily and efficiently in the development process.

Building the whole of VTK or ParaView is just not possible on every single MR/PR.

This is why we think that providing tools for the community in order to be able to build and test ParaView plugins, ParaView based applications, VTK dependent modules and VTK based applications will help all these projects grow. This is especially true since the improvements of the testing framework in both VTK and ParaView in their respective upcoming releases.

Docker is the solution we choose for this and is a de-facto standard in many communities.

So we are very happy to introduce vtk-for-ci and paraview-for-ci dockerhub repositories.

By using these docker images, you will be able to very easily build and test your projects.

To get started, you can have a look at the two examples repositories:

• vtk-ci-example : gitlab / github
• paraview-ci-example : gitlab / github

How to use on gitlab

If you are using GitLab, your .gitlab-ci.yml file may look like:

build-plugin:
  image: kitware/paraview-for-ci:v5.10.1
  stage: build
  script:
    - mkdir -p build
    - cd build
    - cmake -DCMAKE_PREFIX_PATH=/opt/paraview/install/ -DBUILD_TESTING=ON -DCMAKE_BUILD_TYPE=Release ..
    - cmake --build . --parallel 2
  artifacts:
    expire_in: 1h
    when: always
    paths:
      - build/

test-plugin:
  image: kitware/paraview-for-ci:v5.10.1
  stage: test
  script:
    - cd build
    - xvfb-run ctest -j 2 --output-on-failure || xvfb-run ctest -j 1 --rerun-failed -VV 
  dependencies:
    - build-plugin
  needs:
    - build-plugin

How to use on github

If you are using github, your ci.yml file may look like:

name: CI

on:
  pull_request:
    types: [opened, synchronize, reopened, ready_for_review]
  push:
    branches:
      - master

jobs:
  ci:
    if: github.event.pull_request.draft == false

    runs-on: ubuntu-latest
    strategy:
      matrix:
        version: [ v5.9.1, v5.10.1, latest ]
    container: kitware/paraview-for-ci:${{ matrix.version }}

    steps:

    - name: Checkout
      uses: actions/checkout@v2
      with:
        path: 'source'
        fetch-depth: 0
        lfs: 'true'

    - name: Setup Directories
      working-directory: ${{github.workspace}}
      run: mkdir build

    - name: Configure
      working-directory: ${{github.workspace}}/build
      run: cmake ../source -DCMAKE_PREFIX_PATH=/opt/paraview/install/ -DBUILD_TESTING=ON -DCMAKE_BUILD_TYPE=Release

    - name: Build
      working-directory: ${{github.workspace}}/build
      run: cmake --build . --parallel 2

    - name: Test
      working-directory: ${{github.workspace}}/build
      run: xvfb-run ctest -j 2 --output-on-failure || xvfb-run ctest -j 1 --rerun-failed -VV

Technicalities

Graphical testing is supported using Xvfb and mesa.

The ParaView image is built with reasonable configuration that should cover most usecases

cmake -GNinja 
  -DPARAVIEW_USE_PYTHON=ON 
  -DPARAVIEW_USE_MPI=ON 
  -DVTK_SMP_IMPLEMENTATION_TYPE=TBB 
  -DCMAKE_BUILD_TYPE=Release 
  -DCMAKE_INSTALL_PREFIX=/opt/paraview/install 

The VTK image is built with a reasonable configuration that could cover most usecases

cmake -GNinja 
  -DVTK_USE_MPI=ON 
  -DVTK_GROUP_ENABLE_MPI=YES 
  -DVTK_WRAP_PYTHON=ON 
  -DVTK_PYTHON_VERSION=3 
  -DVTK_SMP_ENABLE_OPENMP=ON 
  -DVTK_SMP_ENABLE_SEQUENTIAL=ON 
  -DVTK_SMP_ENABLE_STDTHREAD=ON  
  -DVTK_SMP_IMPLEMENTATION_TYPE=STDThread 
  -DVTK_GROUP_ENABLE_Qt=YES 
  -DVTK_GROUP_ENABLE_Rendering=YES 
  -DVTK_GROUP_ENABLE_StandAlone=YES 
  -DVTK_GROUP_ENABLE_Views=YES 
  -DVTK_GROUP_ENABLE_Web=YES 
  -DVTK_MODULE_ENABLE_VTK_GUISupportQtQuick=NO 
  -DCMAKE_BUILD_TYPE=Release 
  -DCMAKE_INSTALL_PREFIX=/opt/vtk/install

If you or your team need more complex images, do not hesitate to contact Kitware for help.

If you are looking to build a plugin for the paraview.org binary release, you may want to look at this Kitware blog.

Special Thanks to Brad King, Charles Gueunet, Khaled Hadj-tahar and Tom Suchel for their help with this work !