CAPE 2.2 is an important release that substantially reduces the amount of
unlikeable code under the hood. In many cases, the improvements will not be
highly visible to users, which is the intent. The intended benefit of this
restructuring is really to make it easier to implement new features, which is
already paying off on several additions in the works.

New Features

• Status check commands, e.g. pyfun -c now have an option to add
  additional columns as columns in the status table. For example:

  $ pyfun -c --add-cols "user,arch"
  Case Config/Run Directory  Status  Iterations Que CPU Time user   arch
  ---- --------------------- ------- ---------- --- -------- ------ ----
  0    poweroff/m0.5a0.0b0.0 ---     /          .            ddalle rom
  1    poweroff/m0.5a2.0b0.0 ---     /          .            jmeero mil
  2    poweroff/m0.5a0.0b2.0 ---     /          .            ddalle ivy

  Users may also use the --cols option to only select certain columns

  $ cape -c --cols "i,case,progress"
  Case Case Folder  Iterations
  ---- ------------ ----------
  0    m0.5a0.0b0.0 500/1000
  1    m0.5a2.0b0.0 / 
  2    m0.5a0.0b2.0 /

• The various solvers now have an ability to detect "early exits." Some
  solvers run fewer than the requested number of iterations if, for example,
  a specified convergence criteria is met early. Previous versions of CAPE
  in some cases got stuck in a loop trying to rerun such cases, but the new
  capability will reduce PhaseIters to match whatever point the solver
  exited at if it detects a purposeful early exit.

• Tecplot subfigures can now work directly from VTK files. If the user
  defines the option VTKFiles for a "Tecplot" type subfigure, CAPE can
  convert them using cape.gruvoc (see the CAPE 2.1.0 release notes)
  combined with PyVista to convert them to .plt files before proceeding
  with the flow viz image generation.

• On a related note, there is now a cape-vtk2plt executable that can
  convert many VTK files to Tecplot .plt format directly from the command
  line.

• The "Mesh" section of each JSON file now supports new options
  "CopyAsFiles" and "LinkAsFiles". These are in addition to
  "CopyFiles", which enables users to copy files into each case's run
  folder as they are. The "CopyAsFiles" option is a :mod:dict that
  enables users to rename files as they are copied. For example

  "Mesh": {
      "CopyAsFiles": {
          "inputs/mesh/config02.lb8.ugrid": "pyfun.lb8.ugrid"
      }
  }

  will copy the file inputs/mesh/config02.lb8.ugrid into each case folder
  but rename it pyfun.lb8.ugrid in the process.

• A new run matrix variable type, "altitude", has been added and has been
  integrated into the cape.pykes module.

Behavior Changes

• The infrastructure for "DataBook" has been rewritten. Commands like

  $ pyfun --fm
  $ pyfun --ll
  $ pyfun --triqfm

  will continue to work as before (though with some more informative output
  to the command line), and the force & moment data will look almost
  identical. However, the line load data files take a much different form.

  A line load databook for a component called "STACK" for a run matrix
  with cases like poweron/m0.50a0.0 would have created a whole herd of
  files in the old-style line load databooks:

  • ll_STACK.csv: status and conds of which cases have been completed
  • lineload/poweron/m0.50a0.0/LineLoad_STACK.csv: loads for one case
  • lineload/poweron/m0.50a2.0/LineLoad_STACK.csv: loads for one case
  • lineload/poweron/m0.50a4.0/LineLoad_STACK.csv: loads for one case
  • lineload/poweron/m0.70a0.0/LineLoad_STACK.csv: loads for one case
  • ...

  You will now see a single file, ll_STACK.csv, which has all of the data
  in a format that's quite compatible with revision control using git. It
  will have a lot of columns! If you have 10 run matrix variables and you
  select 100 slices for the STACK component, the column count will be
  710. But it works quite well and is much simpler to interact with than the
  previous multiple-file mess.

  For a DataBook component of type "TriqFM", you will see a similar theme,
  though the effect is much less dramatic. Suppose you have a component fin
  that has patches front, top, left, right, and back. In the
  old databooks you would have seen these files:

  • triqfm/triqfm_fin.csv (combined loads on all patches)
  • triqfm/triqfm_fin_front.csv
  • triqfm/triqfm_fin_top.csv
  • triqfm/triqfm_fin_left.csv
  • triqfm/triqfm_fin_right.csv
  • triqfm/triqfm_fin_back.csv

  whereas now you will only see a single file

  • triqfm/triqfm_fin.csv

• Raw files for automated reports (e.g. from pyfun --report) can now be
  either saved in the report/ folder (the previous standard) or directly
  in the case folder (the new default). Users can control this with the new
  Location option on the Reports section. It's defined for each report
  and can be either "case" or "report".

  The upgrades to CAPE reports will result in smaller file sizes and also
  enable caching of text-based subfigures (such as force & moment summary
  tables), which previously had to be regenerated each time a report was
  compiled.

Bugs Fixed

• Several recent versions of CAPE did not support having mixed CSV/JSON
  definitions for run matrix conditions. For example, you might have a run
  matrix definition like this:

  "RunMatrix": {
      "Keys": ["mach", "alpha", "beta", "config"],
      "File": "matrix.csv",
      "Config": "poweroff"
  }

  with the following in matrix.csv:

  # mach, alpha, beta
    0.50, 2.0,  0.0
    0.80, 4.0, -1.0
    0.95, 1.5,  2.0
  

  This is supposed to work, using "poweroff" as the value for config
  for all three cases, but it was broken in CAPE 2.1.

• The limiter option for Cart3D in non-adaptive cases now affects the
  input.cntl files that CAPE creates.

Features Paused

• As a result of the major rewrite of how reports are generated, sweep
  reports are temporarily not functional. They can be reimplemented without
  too much difficulty, but didn't make the priority cut for this release.

This is primarily a bug-fix release to help with running Cart3D, especially for
adaptive runs. The early adaptation cycles often exit early when a certain
amount of residual detection is reached. This causes the number of iterations
run to be unpredictable, which has led to problems with CAPE. This bug fix
largely addresses those issues, but certain use cases with Cart3D are still
not functional for cases where large residual drops occur.

New Features

• The interface for py{x} -c now has more options. For example you can
  show the values of additional columns from the run matrix explicitly.

  This example will show the value of run matrix keys called user and
  arch for each row of the status table, provided they exist.

      $ pyfun -I 3:7 --add-cols "user,arch" -c

  This feature is not fully documented yet and will be discussed more in the
  release of CAPE 2.2.0.

Bugs Fixed

• When running Cart3D, CAPE will now avoid trying to restart aero.csh in
  an infinite loop for certain cases. This seemed to be caused by Cart3D
  terminating early when a target residual is reached (see the nOrders
  option for input.cntl). pycart has now been set to lower the
  PhaseIters option if an adaptive phase looks to have run successfully but
  with fewer iterations than expected.

New Features

• CAPE now includes an unstructured mesh module called cape.gruvoc. It
  contains a mesh conversion capability, for example to convert a UGRID
  volume mesh to a Tecplot volume PLT file, you could run

  .. code-block:: console

  $ python3 -m cape.gruvoc pyfun.lb8.ugrid pyfun.plt --mapbc pyfun.mapbc
  

  It also creates a gruvoc executable so that you could just run

  .. code-block:: console

  $ gruvoc pyfun.lb8.ugrid pyfun.plt --mapbc pyfun.mapbc
  

  Another useful tool is gruvoc report, which summarizes the number of
  nodes, surface tris, tets, etc. in a grid.

  .. code-block:: console

  $ gruvoc report pyfun.lb8.ugrid -h
  

  You can call gruvoc report without the -h to see the raw numbers.
  The -h flag means "human-readable", so for example it will abbreviate
  1234567 to "1.2M".

• The pyfun module now contains the ability to directly read FUN3D
  solutions directly, both instantaneous restart files (.flow) and
  time-averaged solutions (_TAVG.1). An example usage is to convert a
  time-averaged solution to a volume Tecplot PLT file:

  .. code-block:: console

  $ gruvoc convert pyfun.lb8.ugrid pyfun_volume_tavg_timestep1000.plt \
      --tavg pyfun_TAVG.1 --add-mach --add-cp --mapbc pyfun.mapbc
  

  The cape.pyfun.casecntl module conveniently provides some built-in
  "workers" (See the notes on the CAPE 2.0.3 release) to make these
  conversions while FUN3D is running. A common use case is to add the
  following to the "RunControl" section for a pyfun JSON file:

  .. code-block:: javascript

  "WorkerPythonFuncs": [
      {
          "name": "flow2plt",
          "type": "runner"
      },
      {
          "name": "tavg2plt",
          "type": "runner"
      },
      {
          "name": "clean",
          "type": "runner"
      }
  ],
  "PostPythonFuncs": [
      {
          "name": "flow2plt",
          "type": "runner"
      },
      {
          "name": "tavg2plt",
          "type": "runner"
      }
  ]
  

  This code will automatically convert each new pyfun.flow file to
  pyfun_volume_timestep{n}.plt and each new pyfun_TAVG.1 file to
  pyfun_volume_tavg_timestep{n}.plt without requiring FUN3D to do any of
  the output except for writing its usual restart files.

• The cape.gruvoc module also contains tools for creating cut planes and
  performing other data sampling routines by using PyVista
  (https://pyvista.org/). The tavg2x runner for pyfun can be used for
  more data sampling routines; see the documentation of that function for
  more information.

• Checking PBS status is now more efficient and effective. CAPE can now
  automatically detect which user's queue to check based on the owner of a
  job (useful if you care calling cape -c from another user's folder) and
  call to multiple PBS servers if appropriate. (This last example has become
  quite usefule on NASA HPC systems, for example, where the CPU and GPU jobs
  are controlled by distinct and separate PBS servers.)

• Several upgrades have been made to the MPI interface in
  cape.cfdx.cmdgen. These changes allow support for mixed GPU/CPU
  workflows (for example when running refine/three on a GPU job with
  FUN3D) and support additional environments that require more command-line
  arguments (for example the new Grace Hopper systems on the NASA Advanced
  Supercomputing facility).

• Fun3D namelist control in the pyfun JSON file now supports setting
  multiple indices of the same option in an efficient manner. Here's an
  example setting the boundary conditions for a variety of surfaces:

      "Fun3D": {
          "boundary_conditions": {
              "wall_temp_flag": {
                  "1-38": true
              },
              "wall_temperature": {
                  "1-38": -1
              },
              "static_pressure_ratio": {
                  "8": 0.5,
                  "15": 0.75,
                  "18": 0.52
              }
      }

  It will lead to a namelist such as this:

      &boundary_conditions
          wall_temp_flag(1:38) = .true.
          wall_temperature(1:38) = -1
          static_pressure_ratio(8) = 0.5
          static_pressure_ratio(15) = 0.75
          static_pressure_ratio(18) = 0.52
      /
  

  This is generally much more practical than making a list of 38 true
  values for wall_temp_flag and is especially convenient for the
  static_pressure_ratio in this example.

Behavior Changes

• Many of the "DataBook" classes in cape.cfdx.databook have been renamed.
  These may cause issues for advanced users who have custom "hooks" or other
  Python modules.

• The default data type in cape.dkit.textdata has been changed to
  int32. This minor change makes it much easier to read CAPE run matrix
  files as DataKits without the need for extra keyword arguments.

• Job status (e.g. when running pyover -c) is now computed by
  CaseRunner instead of Cntl. This is generally more efficient (users
  may notice the difference) and allows stand-alone cases to be aware of
  their status without being part of a run matrix.

Bug Fixes

• The CAPE 2.0.3 pycart modules contained several bugs that prevented
  adaptive runs (even the published CAPE examples) from running properly. All
  CAPE test cases run properly now.

New Features

• CAPE now includes PreShellCmds to go alongise PostShellCmds. This is an
  option in the "RunControl" section that allows the user to run one or
  more BASH (or whatever your shell of choice is) commands prior to running
  the primary CFD solver executables.

• There is an exciting new feature called WorkerShellCmds in the
  "RunControl" section. It allows you to specify 0 or more BASH commands
  that you run every WorkerSleepTime seconds (default=10.0) while your
  case is running. It has working clean-up after the main executable is
  finished, allowing up to "WorkerTimeout" (default=600.0) seconds
  for the last instance of the worker to complete.

• These run hooks also have Python function versions, in the form of options
  PrePythonFuncs, PostPythonFuncs, and WorkerPythonFuncs. If these are
  defined as a simple string, CAPE will import any modules implied by the
  function name and then call that function with no arguments. However, users
  may also specify more details for Python functions by defining the function
  in a dict.

    "RunControl": {
        "WorkerPythonFuncs": [
            "mymod.mufunc",
            {
                "name": "clean",
                "type": "runner"
            },
            {
                "name": "mymod.otherfunc",
                "args": [
                    "$runner",
                    "$mach"
                ]
            }
        ]
    }

• Users of FUN3D and Kestrel can now link the mesh file into folders instead
  of copying it. Set "LinkMesh" to true in the "Mesh" section.

• cape.pyfun in particular changes how it uses XML or JSON configuration
  files (which is specified in the "Config" > "File" setting). In
  previous versions of CAPE, the face labels or component ID numbers in that
  file had to match your actual grid, which had to match your .mapbc
  file. Now CAPE only uses the text names in the ConfigFile, and it's ok to
  include components that aren't actually present in your grid. If your case
  worked as expected before, it will still work now, but for new cases it
  might be much easier to set up. The (new) recommended process is to use a
  ConfigJSON file and only specify a "Tree" section.

Behavior Changes

• Binary files storing iterative histories are no longer saved automatically
• Calculation of job status, especially for FUN3D, is much faster. This
  change should not cause any functional changes for users.
• Python modules used to define hooks are no longer universally imported
  during Cntl instantiation. Modules are imported dynamically if needed
  to execute a hook. The "Modules" setting is still present in the JSON
  file but has no effect.

Bugs Fixed

• Fix bug in area-weighted node normal calculation,
  cape.trifile.TriBase.GetNodeNormals().
• The refine/three capability with FUN3D now works more reliably.

New Features

• New capabilities in nProc, the number of MPI processes to use. Instead of
  requiring a positive integer, there are now four ways to interpret this
  setting:

  • positive integer: "nProc": 128 will continue to work in the
    obvious way that it always has
  • negative integer: "nProc": -2 on a node with 128 cores will mean
    using 126 cores
  • fraction: "nProc": 0.5 will mean using 50% (rounded down), so
    mpiexec -np 64 on a 128-core node
  • blank: "nProc": null (or leaving out entirely) means use all the
    MPI procs available

Behavior Changes

• Don't write Archive settings to each case folder

Bugs Fixed

All of the tutorials at

https://github.com/nasa-ddalle/

now work properly with this version. Most of the updates were to the tutorials
themselves, but some CAPE bugs were fixed, too.

New Features

• GPU options in RunControl section of options
• CaseRunner system calls now allow piping lines of a file to STDIN

Behavior Changes

• Archiving uses tar -u if using the standard .tar archive format
• Fix -e option to execute commands in case folders, and allow it to run
  regular system commands (not just local scripts)

Bugs Fixed

• Add several missing options to RunMatrix definitions
• Fix zone type when reading Cart3D .triq files into Tecplot format
• Improve handling of different-sized iterative histories in CaseFM
• Add PyYAML and colorama to install requirements

New Features

• Added a command cape --1to2 to help update Python files written against
  the CAPE 1.2 API to the newer module names mentioned below.

• The main input file can now be a YAML file in addition to the standard
  JSON. However, there is no "include" statement like the JSONFile()
  directive supported in CAPE JSON files.

• New command-line interface. The CLI supports the commands that would have
  worked for CAPE 1 but also support a new method that allows the user to be
  more explicit about the primary purpose of the command. For example

      $ pyfun --re "m1.2" --report

  is the same as

      $ pyfun report --re "m1.2"

  The new CLI also implements checks so that misspelled or unrecognized
  options will result in an error instead of just ignoring those options.

• Created a new executable cape-tec that takes a Tecplot(R) layout file
  as input and exports a PNG from that layout.

• Rewritten interface to RunControl > Archive. Users may now prescribe
  "only keep the most recent file of this set" of multiple patterns in a
  single line. For example ...

      "Archive": {
          "SearchMethod": "regex",
          "clean": {
              "PreDeleteFiles": {
                  "pyfun[0-9]+_([a-z][a-z0-9_-]+)_timestep[0-9]+\\.plt": 1
              }
          }
      }

  This will delete most Tecplot .plt files but keep the most recent 1
  matches. The new feature is that it will collect all the files that match
  this regular expression but divide them into separate lists for all the
  unique values of the regular expression group (the part inside
  parentheses). So if you have the following files

  • pyfun00_plane-y0_timestep1000.plt
  • pyfun00_tec_boundary_timestep1000.plt
  • pyfun01_plane-y0_timestep2000.plt
  • pyfun01_tec_boundary_timestep2000.plt
  • pyfun02_plane-y0_timestep3000.plt
  • pyfun02_plane-y0_timestep4000.plt
  • pyfun02_tec_boundary_timestep3000.plt
  • pyfun02_tec_boundary_timestep4000.plt

  Then it would delete most of these files but only keep

  • pyfun02_plane-y0_timestep4000.plt
  • pyfun02_tec_boundary_timestep4000.plt

  This would not have been possible in CAPE 1; users would need to provide
  two separate instructions.

• A RunMatrix key with the type "translation" can now use two named
  points as the "Vector". This means that the direction that a component
  is translated can be affected by prior RunMatrix keys

Behavior Changes

• Many modules have been renamed, including renaming the case modules to
  the less-confusing name casecntl. In addition, the main cntl module
  has been moved into the cape.cfdx folder.

Bugs Fixed

• Determination of number of available MPI ranks on Slurm jobs

New Features

• A RunMatrix key with the type "translation" can now use two named
  points as the "Vector". This means that the direction that a component
  is translated can be affected by prior RunMatrix keys

Bugs Fixed

• Determination of number of available MPI ranks on Slurm jobs

CAPE 1.2.1

New Features

• Each case now generates logs, which are helpful for debugging or just
  understanding the sequence of actions CAPE takes. The two log files within
  each case are cape/cape-main.log and cape/cape-verbose.log).
• PBS/Slurm job names are now longer (32 chars instead of 15), and the length
  is configurable (RunMatrix > MaxJobNameLength).

Behavior Changes

• PBS/Slurm job IDs are now saved as the full string instead of just the
  job number (often something like 123456.pbspl1)

Bugs Fixed

• Better support of newer aero.csh script for Cart3D

CAPE 1.2 is a smaller change than CAPE 1.1 and focuses on improving the quality
of CAPE's underlying code. Many modules have been de-linted, and some of the
older modules have been rewritten. Test coverage is also significantly
improved.

New Features

• The iterative histories (both CaseFM and CaseResid) now create a
  cache file so that CAPE can read them in much faster after the first read.
  It also creates a uniform file format for users who might be interested in
  saving iterative histories.
• Add TSVTecDatFile class to read Tecplot-style column-data into
  DataKit. See
  https://nasa.github.io/cape-doc/1.2/api/attdb/ftypes/tecdatfile.html
• Add a --incremental option (or set RunControl ▸ StartNextPhase to
  False) option to run one phase at a time. See
  https://nasa.github.io/cape-doc/1.2/common/json/RunControl.html for the
  StartNextPhase option and/or
  https://nasa.github.io/cape-doc/1.2/bin/pyfun.html for --incremental.

Behavior Changes

• The iterative history modules, CaseFM and CaseResid, are now
  subclasses of DataKit. Among other things, this means that what used to
  be fm.CN is now fm["CN"]. This is a major improvement to making
  those classes extensible for histories of things other than forces &
  moments.

• The cape.filecntl.filecntl module, which is critical to how CAPE
  reads and modifies CFD input files, was rewritten and tested to 100%
  coverage.

• Rename some RunControl options to more understandable

  • Resubmit ➝ ResubmitNextPhase
  • Continue ➝ opposite of ResubmitSamePhase

  (See https://nasa.github.io/cape-doc/1.2/common/json/RunControl.html)