Skip to content

support char arrays and complicated dtypes (structs) - #149

Merged
SylvainCorlay merged 2 commits into
xtensor-stack:masterfrom
wolfv:add_char_support
Jun 20, 2018
Merged

SylvainCorlay merged 2 commits into
xtensor-stack:masterfrom
wolfv:add_char_support

Conversation

@wolfv

@wolfv wolfv commented May 24, 2018

Copy link
Copy Markdown
Member

This adds support for char arrays:

arr = np.array(["hallo", "test", "123123"], dtype=np.dtype('|S20'))
void test_chars(xt::pyarray<char[20]>& arr)
{
    std::cout << arr(0) << std::endl;
}

It also adds some initial dtype support (i.e. you can declare structs, use them inside C++ and serialize them from numpy). However, pybind has a lot of code that deals with padding differences from C++ / NumPy which I haven't really looked at.

This code also removes our own typenum stuff as well as switching to use PyArray_EquivTypes instead of comparing the typenums. There is a good chance that comparing typenums is faster. Therefore, we could statically enable typenums if we know (from the C++ type) that a statically determinable typenum is available.

@iamthebot you might be interested in this based off of your question in #142

@wolfv

wolfv commented May 25, 2018

Copy link
Copy Markdown
Member Author

The way it's implemented now, it falls back to the typenum check for arithmetic types, and checks dtypes for more complicated types (like arrays, or structs).

We reuse Pybind11s mechanism to bind struct arrays but that's experimental for now.

@wolfv wolfv changed the title add initial support for char arrays support char arrays and complicated dtypes (structs) Jun 20, 2018
@wolfv
wolfv force-pushed the add_char_support branch 2 times, most recently from cf2c721 to 03ecfcc Compare June 20, 2018 18:53
@wolfv
wolfv force-pushed the add_char_support branch from 03ecfcc to d036ecb Compare June 20, 2018 18:56
Comment thread test_python/main.cpp
xt::pyarray<B> dtype_from_python(xt::pyarray<B>& b)
{
if (b(0).a != 1 || b(0).b != 'p' || b(1).a != 123 || b(1).b != 'c')
throw std::runtime_error("FAIL");

@SylvainCorlay SylvainCorlay Jun 20, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{}!

Comment thread include/xtensor-python/pycontainer.hpp Outdated
template <class T>
bool check_array(const pybind11::handle& src)
{
using is_arithmetic_type = std::integral_constant<bool, !!pybind11::detail::satisfies_any_of<T, std::is_arithmetic, xtl::is_complex>::value>;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we convert to bool instead of doing !!?

Comment thread test_python/test_pyarray.py Outdated
import os
import sys
import subprocess
import gc

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..

Comment thread test_python/main.cpp Outdated
****************************************************************************/

#include <numeric>
#include <limits>

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

..

Comment thread .travis.yml Outdated
# Useful for debugging any issues with conda
- conda info -a
- conda install pytest numpy pybind11==2.2.1 -c conda-forge
- conda install pytest numpy pybind11==2.2.3 -c conda-forge

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need 2.2.3, (i.e. do we need to update the readme?)

@SylvainCorlay
SylvainCorlay merged commit 1d6429f into xtensor-stack:master Jun 20, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants