Skip to content

Commit 60b8df3

Browse files
yfsn666yfsn666
authored
[INLONG-12012][SDK] Put the Python Dataproxy SDK target directory in the command line parameters to avoid interruption of the build process (#12017)
* [INLONG-12012][SDK] Put the Python Dataproxy SDK target directory in the command line parameters to avoid interruption of the build process * fix comments
1 parent f54b557 commit 60b8df3

File tree

2 files changed

+63
-39
lines changed

2 files changed

+63
-39
lines changed

‎inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-python/README.md‎

Lines changed: 21 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -48,17 +48,32 @@ Go to the `dataproxy-sdk-python` root directory, and run the following commands:
4848

4949
```bash
5050
chmod +x ./build.sh
51-
./build.sh
51+
./build.sh [target_directory]
5252
```
53-
When the .so file is generated, you will see the following message, you can choose to enter the target directory for the .so files. By default, the .so file will be copied to the system python site-packages directory:
5453

55-
```txt
56-
Your system's Python site-packages directory is: xxx/xxx
57-
Enter the target directory for the .so files (Press Enter to use the default site-packages directory):
58-
```
54+
**Parameters:**
55+
- `target_directory` (optional): Directory to install .so files. If not provided, the .so files will be copied to the system Python site-packages directories automatically.
56+
57+
**Usage Examples:**
58+
59+
1. **Install to system site-packages (default behavior):**
60+
```bash
61+
./build.sh
62+
```
63+
The script will automatically detect and copy .so files to all available Python site-packages directories.
64+
65+
2. **Install to a specific directory:**
66+
```bash
67+
./build.sh /path/to/your/target/directory
68+
```
69+
The script will copy .so files to the specified directory. Make sure the target directory exists before running the command.
5970

6071
After the build process finished, you can import the package (`import inlong_dataproxy`) in your python project to use InLong dataproxy.
6172

73+
**Important Notes:**
74+
- If you specify a target directory, make sure it exists before running the build script. The script will exit with an error if the specified directory doesn't exist.
75+
- If no target directory is specified and no system site-packages directories are found, the .so files will remain in the `build` directory and you'll need to copy them manually to your project.
76+
6277
> **Note**: When the C++ SDK or the version of Python you're using is updated, you'll need to rebuild it by the above steps.
6378
6479
## Config Parameters

‎inlong-sdk/dataproxy-sdk-twins/dataproxy-sdk-python/build.sh‎

Lines changed: 42 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
#!/bin/bash
12
#
23
# Licensed to the Apache Software Foundation (ASF) under one or more
34
# contributor license agreements. See the NOTICE file distributed with
@@ -16,10 +17,24 @@
1617
# Initialize the configuration files of inlong components
1718
#
1819

19-
#!/bin/bash
20-
2120
set -e
2221

22+
# Parse command line arguments
23+
TARGET_DIR=""
24+
if [ $# -eq 1 ]; then
25+
TARGET_DIR="$1"
26+
echo "Using user-specified target directory: $TARGET_DIR"
27+
# Check if the target directory exists early to avoid wasting compilation time
28+
if [ ! -d "$TARGET_DIR" ]; then
29+
echo "Error: Target directory '$TARGET_DIR' does not exist!"
30+
exit 1
31+
fi
32+
elif [ $# -gt 1 ]; then
33+
echo "Usage: $0 [target_directory]"
34+
echo " target_directory: Optional. Directory to install .so files. If not provided, will use system site-packages directories."
35+
exit 1
36+
fi
37+
2338
BASE_DIR=$(dirname "$0")
2439
PY_SDK_DIR=$(cd "$BASE_DIR"; pwd)
2540

@@ -73,8 +88,8 @@ if [ ! -d "$PY_SDK_DIR/pybind11/build" ]; then
7388
trap 'echo "Error occurred during pybind11 build. Deleting pybind11 folder..."; cd $PY_SDK_DIR; rm -r pybind11; exit 1' ERR
7489

7590
cmake "$PY_SDK_DIR/pybind11"
76-
cmake --build "$PY_SDK_DIR/pybind11/build" --config Release --target check
77-
make -j 4
91+
cmake --build "$PY_SDK_DIR/pybind11/build" --config Release
92+
make -j"$(nproc)"
7893

7994
# Remove the trap command if the build is successful
8095
trap - ERR
@@ -105,37 +120,31 @@ if [ -d "$PY_SDK_DIR/build" ]; then
105120
fi
106121
mkdir "$PY_SDK_DIR/build" && cd "$PY_SDK_DIR/build"
107122
cmake "$PY_SDK_DIR"
108-
make -j 4
109-
110-
# Get all existing Python site-packages directories
111-
SITE_PACKAGES_DIRS=($(python -c "import site,os; print(' '.join([p for p in site.getsitepackages() if os.path.isdir(p)]))"))
112-
113-
# Check if the SITE_PACKAGES_DIRS array is not empty
114-
if [ ${#SITE_PACKAGES_DIRS[@]} -ne 0 ]; then
115-
# If not empty, display all found site-packages directories to the user
116-
echo "Your system's existing Python site-packages directories are:"
117-
for dir in "${SITE_PACKAGES_DIRS[@]}"; do
118-
echo " $dir"
119-
done
120-
else
121-
# If empty, warn the user and prompt them to enter the target directory in the next step
122-
echo "Warn: No existing site-packages directories found, please enter the target directory for the .so files in the following step!"
123-
fi
123+
make -j"$(nproc)"
124124

125-
# Prompt user for the target directory for .so files
126-
read -r -p "Enter the target directory for the .so files (Press Enter to use all above site-packages directories): " target_dir
127-
128-
# If user input is empty, use all found site-packages directories
129-
if [ -z "$target_dir" ]; then
130-
for dir in "${SITE_PACKAGES_DIRS[@]}"; do
131-
echo "Copying .so files to $dir"
132-
# Find all .so files in $PY_SDK_DIR/build and copy them to the current site-packages directory
133-
find "$PY_SDK_DIR/build" -name "*.so" -print0 | xargs -0 -I {} cp {} "$dir"
134-
done
125+
# Handle installation based on command line arguments
126+
if [ -n "$TARGET_DIR" ]; then
127+
# User specified a target directory via command line argument
128+
echo "Copying .so files to user-specified directory: $TARGET_DIR"
129+
find "$PY_SDK_DIR/build" -name "*.so" -print0 | xargs -0 -I {} cp {} "$TARGET_DIR"
135130
else
136-
# If user specified a directory, copy .so files there
137-
echo "Copying .so files to $target_dir"
138-
find "$PY_SDK_DIR/build" -name "*.so" -print0 | xargs -0 -I {} cp {} "$target_dir"
131+
# No command line argument provided, use system site-packages directories
132+
# Get all existing Python site-packages directories
133+
SITE_PACKAGES_DIRS=($(python -c "import site,os; print(' '.join([p for p in site.getsitepackages() if os.path.isdir(p)]))"))
134+
if [ ${#SITE_PACKAGES_DIRS[@]} -ne 0 ]; then
135+
echo "No target directory specified, using system site-packages directories:"
136+
for dir in "${SITE_PACKAGES_DIRS[@]}"; do
137+
echo " $dir"
138+
done
139+
for dir in "${SITE_PACKAGES_DIRS[@]}"; do
140+
echo "Copying .so files to $dir"
141+
# Find all .so files in $PY_SDK_DIR/build and copy them to the current site-packages directory
142+
find "$PY_SDK_DIR/build" -name "*.so" -print0 | xargs -0 -I {} cp {} "$dir"
143+
done
144+
else
145+
echo "Error: No system site-packages directories found and no target directory specified!"
146+
echo "The .so file is located in $PY_SDK_DIR/build, you can copy it manually to your project"
147+
fi
139148
fi
140149

141150
# Clean the cpp dataproxy directory

0 commit comments

Comments
 (0)