How to Change the Port? #3034

ytooyama · 2024-11-27T00:53:25Z

ytooyama
Nov 27, 2024

Thank you as always.

Feldera uses the 8080 port.
However, I am already using the 8080 port elsewhere.
I want to change the port used by Feldera.

Next, how should I reflect the correction of --allowed-origins "http://localhost:xyz"?

ports:
# If you change the host side of the port mapping here, don't forget to
# also add a corresponding --allowed-origins argument to the pipeline
# manager command below (e.g., --allowed-origins "http://localhost:xyz").
# Otherwise, browsers will complain about CORS errors
- "28080:8080"
- "28081:8081"

Should I create a Docker file like this and remake the image?

FROM ghcr.io/feldera/pipeline-manager:0.30.0

# Run the precompile phase to speed up Rust compilations during deployment
RUN ./pipeline-manager --bind-address=0.0.0.0 --sql-compiler-home=/home/feldera/lib/sql-to-dbsp-compiler --dbsp-override-path=/home/feldera/lib --precompile
ENV BANNER_ADDR=localhost
ENTRYPOINT ["./pipeline-manager", "--bind-address=0.0.0.0", "--sql-compiler-home=/home/feldera/lib/sql-to-dbsp-compiler", "--dbsp-override-path=/home/feldera/lib", "--allowed-origins", "https://www.feldera.com", "--allowed-origins", "http://localhost:28080", "--demos-dir", "/home/feldera/demos"]

Or should I use this?

https://github.com/feldera/feldera/blob/main/deploy/Dockerfile

Thank you.

Answered by lalithsuresh

Nov 27, 2024

You can edit the pipeline-manager service in the compose file to add the argument and edit the ports field:

diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml
index 7e4a14872..68085b17a 100644
--- a/deploy/docker-compose.yml
+++ b/deploy/docker-compose.yml
@@ -8,8 +8,11 @@ services:
       # also add a corresponding --allowed-origins argument to the pipeline
       # manager command below (e.g., --allowed-origins "http://localhost:xyz").
       # Otherwise, browsers will complain about CORS errors
-      - "8080:8080"
-      - "8081:8081"
+      - "28080:8080"
+      - "28081:8081"
+    command:
+      - --allowed-origins
+      - "http://localhost:28080"
     stop_grace_p…

View full answer

ytooyama · 2024-11-27T01:36:38Z

ytooyama
Nov 27, 2024
Author

I tried to edit like this and run docker compose up -d, but it failed.

name: feldera
services:
  pipeline-manager:
    tty: true
    build:
      context: ./feldera-main
      dockerfile: deploy/Dockerfile
    ports:
      # If you change the host side of the port mapping here, don't forget to
      # also add a corresponding --allowed-origins argument to the pipeline
      # manager command below (e.g., --allowed-origins "http://localhost:xyz").
      # Otherwise, browsers will complain about CORS errors
      - "28080:8080"
      - "28081:8081"
    stop_grace_period: 0s
    environment:
      - RUST_LOG=info,actix_web=error,tokio_postgres=info
      - RUST_BACKTRACE=1
      - AUTH_CLIENT_ID
      - AUTH_ISSUER
    security_opt:
      # The default seccomp profile disables the `io_uring_*` syscalls that we need.
      # Ideally, we would minimally relax the default profile, which is usually in
      # /usr/share/container/seccomp.json, but we often start via `curl` of just this
      # `docker-compose.yml`, so there's no way to grab the specific file at the
      # same time, so instead we just use `seccomp:unconfined`.
      #
      #- seccomp:seccomp.json
      - seccomp:unconfined
    healthcheck:
      # TODO: add `/status` endpoint.
      test:
        [
          "CMD-SHELL",
          "curl --fail --request GET --url http://localhost:8080/healthz"
        ]
      interval: 10s
      timeout: 5s
      retries: 5
    cpus: "0.5"
    mem_reservation: "512M"
    mem_limit: "2048M"

And edit the ENTRYPOINT...

# The base image contains tools to build the code given that
# we need a Java and Rust compiler to run alongside the pipeline manager
# as of now. This will change later.
FROM ubuntu:24.04 AS base
ENV DEBIAN_FRONTEND=noninteractive
# These two environment variables are used to make openssl-sys pick
# up libssl-dev and statically link it. Without it, our build defaults
# to building a vendored version of OpenSSL.
ENV OPENSSL_NO_VENDOR=1
ENV OPENSSL_STATIC=1
RUN apt update --fix-missing && apt install \
  # pkg-config is required for cargo to find libssl
  libssl-dev pkg-config \
  # rdkafka dependency needs cmake and a CXX compiler
  cmake build-essential \
  # To install rust
  curl  \
  # For running the SQL compiler
  openjdk-21-jre-headless -y \
  # Install locale-gen
  locales \
  # To add the nodesource debian repository
  ca-certificates gnupg \
  # Required by the `metrics-exporter-tcp` crate
  protobuf-compiler

# Set UTF-8 locale. Needed for the Rust compiler to handle Unicode column names.
RUN sed -i -e 's/# en_US.UTF-8 UTF-8/en_US.UTF-8 UTF-8/' /etc/locale.gen && \
  locale-gen
ENV LC_ALL=en_US.UTF-8
ENV LANG=en_US.UTF-8
ENV LANGUAGE=en_US:en

# Use cargo-chef to produce a recipe.json file
# to cache the requisite dependencies
FROM base AS chef
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
RUN /root/.cargo/bin/cargo install cargo-chef
WORKDIR app

# Build web-ui
FROM base AS web-ui-builder

## Install Bun.js
RUN apt install unzip -y
RUN curl -fsSL https://bun.sh/install | bash
ENV PATH="$HOME/.bun/bin:$PATH"
RUN ln -s $HOME/.bun/bin/bun /usr/bin/bun

# sveltekit
COPY web-console/package.json web-console/
COPY web-console/bun.lockb web-console/
RUN cd web-console && bun install

COPY web-console/static web-console/static
COPY web-console/src web-console/src
COPY web-console/.prettierignore web-console/
COPY web-console/.prettierrc web-console/
COPY web-console/eslint.config.js web-console/
COPY web-console/postcss.config.js web-console/
COPY web-console/svelte.config.js web-console/
COPY web-console/tailwind.config.ts web-console/
COPY web-console/tsconfig.json web-console/
COPY web-console/vite.config.ts web-console/
RUN cd web-console && bun run build

# Cache dependencies from rust
FROM chef AS planner
COPY Cargo.toml Cargo.toml
COPY Cargo.lock Cargo.lock
COPY crates crates
RUN mkdir sql-to-dbsp-compiler
COPY sql-to-dbsp-compiler/lib sql-to-dbsp-compiler/lib
RUN /root/.cargo/bin/cargo chef prepare --recipe-path recipe.json

# Use the recipe.json file to build dependencies first and cache that
# layer for faster incremental builds of source-code only changes
FROM chef AS builder
ENV CARGO_INCREMENTAL=0
COPY --from=planner /app/recipe.json recipe.json
RUN /root/.cargo/bin/cargo chef cook --release --recipe-path recipe.json --bin=pipeline-manager --features=pg-embed
COPY Cargo.toml Cargo.toml
COPY Cargo.lock Cargo.lock
COPY crates crates
RUN mkdir sql-to-dbsp-compiler || true
COPY sql-to-dbsp-compiler/lib sql-to-dbsp-compiler/lib
COPY --from=web-ui-builder /web-console/build web-console-build
ENV WEBUI_BUILD_DIR=/app/web-console-out
ENV WEBCONSOLE_BUILD_DIR=/app/web-console-build
RUN /root/.cargo/bin/cargo build --release --bin=pipeline-manager --features=pg-embed

# Java build can be performed in parallel
FROM base AS javabuild
RUN apt install maven git -y
RUN mkdir sql
COPY sql-to-dbsp-compiler /sql/sql-to-dbsp-compiler
RUN --mount=type=cache,target=/root/.m2 \
    cd /sql/sql-to-dbsp-compiler && ./build.sh

# Minimal image for running the pipeline manager
FROM base AS release
# Pipeline manager binary
RUN useradd -ms /bin/bash feldera
USER feldera
WORKDIR /home/feldera
COPY --from=builder /app/target/release/pipeline-manager pipeline-manager
# SQL compiler uber jar
RUN mkdir -p lib/sql-to-dbsp-compiler/SQL-compiler/target
COPY --from=javabuild /sql/sql-to-dbsp-compiler/SQL-compiler/target/sql2dbsp-jar-with-dependencies.jar lib/sql-to-dbsp-compiler/SQL-compiler/target/sql2dbsp-jar-with-dependencies.jar

# Reuse `Cargo.lock` to ensure consistent crate versions.
RUN mkdir -p .feldera/cargo_workspace
COPY --chown=feldera Cargo.lock .feldera/cargo_workspace/Cargo.lock

# Copy over demos
RUN mkdir -p demos
COPY demo/packaged/sql demos

# The crates needed for the SQL compiler
COPY crates/ lib/crates/
COPY README.md lib/README.md
RUN mkdir -p lib/sql-to-dbsp-compiler/lib

# Copy over the rust code and sql-to-dbsp script
COPY sql-to-dbsp-compiler/lib lib/sql-to-dbsp-compiler/lib
COPY sql-to-dbsp-compiler/temp lib/sql-to-dbsp-compiler/temp
COPY sql-to-dbsp-compiler/SQL-compiler/sql-to-dbsp lib/sql-to-dbsp-compiler/SQL-compiler/sql-to-dbsp

# Install cargo and rust for this non-root user
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal
# The download URL for mold uses x86_64/aarch64 whereas dpkg --print-architecture says amd64/arm64
RUN arch=`dpkg --print-architecture | sed "s/arm64/aarch64/g" | sed "s/amd64/x86_64/g"`; \
  curl -LO https://github.com/rui314/mold/releases/download/v2.32.1/mold-2.32.1-$arch-linux.tar.gz \
  && tar -xzvf mold-2.32.1-$arch-linux.tar.gz \
  && mv mold-2.32.1-$arch-linux /home/feldera/mold \
  && rm mold-2.32.1-$arch-linux.tar.gz
ENV PATH="$PATH:/home/feldera/.cargo/bin:/home/feldera/mold/bin"
ENV RUSTFLAGS="-C link-arg=-fuse-ld=mold"
# Run the precompile phase to speed up Rust compilations during deployment
RUN ./pipeline-manager --bind-address=0.0.0.0 --sql-compiler-home=/home/feldera/lib/sql-to-dbsp-compiler --dbsp-override-path=/home/feldera/lib --precompile
ENV BANNER_ADDR=localhost
ENTRYPOINT ["./pipeline-manager", "--bind-address=0.0.0.0", "--sql-compiler-home=/home/feldera/lib/sql-to-dbsp-compiler", "--dbsp-override-path=/home/feldera/lib", "--allowed-origins", "https://www.feldera.com", "--allowed-origins", "http://localhost:8080", "--allowed-origins", "http://localhost:28080","--demos-dir", "/home/feldera/demos"]

error logs:

 => ERROR [pipeline-manager web-ui-builder 17/17] RUN cd web-console && bun run build                                                                              237.0s
------
 > [pipeline-manager web-ui-builder 17/17] RUN cd web-console && bun run build:
0.439 $ vite build
1.225 ▲ [WARNING] Cannot find base config file "./.svelte-kit/tsconfig.json" [tsconfig.json]
1.225 
1.225     tsconfig.json:2:13:
1.225       2 │   "extends": "./.svelte-kit/tsconfig.json",
1.225         ╵              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1.225 
5.973 vite v5.4.11 building SSR bundle for production...
7.542 transforming...
9.091 1:25:36 AM [vite-plugin-svelte] src/lib/components/pipelines/PipelineNameInput.svelte:32:6 `createPipelineInputRef` is updated, but is not declared with `$state(...)`. Changing its value will not correctly trigger updates
9.091 30:   })
9.091 31: 
9.091 32:   let createPipelineInputRef: HTMLElement
9.091                                 ^
9.091 33: 
9.091 34:   function clickOutsideFocus(element: HTMLElement, callbackFunction: () => void) {
9.113 1:25:36 AM [vite-plugin-svelte] src/lib/components/auth/AuthPopupMenu.svelte:45:4 Buttons and links should either contain text or have an `aria-label` or `aria-labelledby` attribute
9.113 43:     >
9.113 44: 
9.113 45:     <button
9.113                ^
9.113 46:       onclick={toggleDarkMode}
9.113 47:       class="preset-grayout-surface btn-icon text-[24px]
9.563 1:25:36 AM [vite-plugin-svelte] src/lib/components/pipelines/list/Actions.svelte:117:2 Buttons and links should either contain text or have an `aria-label` or `aria-labelledby` attribute
9.563 115: 
9.563 116: {#snippet _delete()}
9.563 117:   <button
9.563               ^
9.563 118:     class="{buttonClass} {shortClass} fd fd-delete bg-surface-50-950 {iconClass}"
9.563 119:     onclick={() => (globalDialog.dialog = deleteDialog)}
9.563 1:25:36 AM [vite-plugin-svelte] src/lib/components/pipelines/list/Actions.svelte:250:2 Buttons and links should either contain text or have an `aria-label` or `aria-labelledby` attribute
9.564 248: {/snippet}
9.564 249: {#snippet _configureResources()}
9.564 250:   <button
9.564               ^
9.564 251:     onclick={() => (globalDialog.dialog = resourcesDialog)}
9.564 252:     class="{buttonClass} {shortClass} fd fd-settings_input_component bg-surface-50-950 {iconClass}"
9.565 1:25:36 AM [vite-plugin-svelte] src/lib/components/pipelines/list/Actions.svelte:262:2 Buttons and links should either contain text or have an `aria-label` or `aria-labelledby` attribute
9.565 260: {/snippet}
9.565 261: {#snippet _configureProgram()}
9.565 262:   <button
9.565               ^
9.565 263:     onclick={() => (globalDialog.dialog = compilationDialog)}
9.565 264:     class="{buttonClass} {shortClass} fd fd-settings bg-surface-50-950 {iconClass}"
236.9 error: script "build" was terminated by signal SIGKILL (Forced quit)
236.9 Killed
------
failed to solve: process "/bin/sh -c cd web-console && bun run build" did not complete successfully: exit code: 137

0 replies

mihaibudiu · 2024-11-27T01:39:28Z

mihaibudiu
Nov 27, 2024
Maintainer

the errors you report seem unrelated to your change.

0 replies

ytooyama · 2024-11-27T01:52:18Z

ytooyama
Nov 27, 2024
Author

The method of starting using the image that was rewritten from the base image went well.

FROM ghcr.io/feldera/pipeline-manager:0.30.0

# Run the precompile phase to speed up Rust compilations during deployment
RUN ./pipeline-manager --bind-address=0.0.0.0 --sql-compiler-home=/home/feldera/lib/sql-to-dbsp-compiler --dbsp-override-path=/home/feldera/lib --precompile
ENV BANNER_ADDR=localhost
ENTRYPOINT ["./pipeline-manager", "--bind-address=0.0.0.0", "--sql-compiler-home=/home/feldera/lib/sql-to-dbsp-compiler", "--dbsp-override-path=/home/feldera/lib", "--allowed-origins", "https://www.feldera.com", "--allowed-origins", "http://localhost:28080", "--demos-dir", "/home/feldera/demos"]

I was able to access.

% fda --host http://localhost:28080 pipelines
╭──────┬──────────╮
│ name │ status   │
├──────┼──────────┤
│ p1   │ Shutdown │
╰──────┴──────────╯

0 replies

lalithsuresh · 2024-11-27T01:54:19Z

lalithsuresh
Nov 27, 2024
Maintainer

Hi @ytooyama. Glad you were able to resolve it. You could also add "--allowed-origins http://localhost:28080" to the arguments. For example:

docker run -p 28080:8080 --tty --rm -it ghcr.io/feldera/pipeline-manager:0.30.0 --allowed-origins http://localhost:28080

3 replies

ytooyama Nov 27, 2024
Author

Thank you. By the way, how do I run "docker compose"?

lalithsuresh Nov 27, 2024
Maintainer

You can edit the pipeline-manager service in the compose file to add the argument and edit the ports field:

diff --git a/deploy/docker-compose.yml b/deploy/docker-compose.yml
index 7e4a14872..68085b17a 100644
--- a/deploy/docker-compose.yml
+++ b/deploy/docker-compose.yml
@@ -8,8 +8,11 @@ services:
       # also add a corresponding --allowed-origins argument to the pipeline
       # manager command below (e.g., --allowed-origins "http://localhost:xyz").
       # Otherwise, browsers will complain about CORS errors
-      - "8080:8080"
-      - "8081:8081"
+      - "28080:8080"
+      - "28081:8081"
+    command:
+      - --allowed-origins
+      - "http://localhost:28080"
     stop_grace_period: 0s
     environment:
       - RUST_LOG=info,actix_web=error,tokio_postgres=info

Answer selected by ytooyama

ytooyama Nov 27, 2024
Author

Thank you.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to Change the Port? #3034

Uh oh!

{{title}}

Uh oh!

Replies: 4 comments 3 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

How to Change the Port? #3034

Uh oh!

ytooyama Nov 27, 2024

Replies: 4 comments · 3 replies

Uh oh!

ytooyama Nov 27, 2024 Author

Uh oh!

mihaibudiu Nov 27, 2024 Maintainer

Uh oh!

ytooyama Nov 27, 2024 Author

Uh oh!

lalithsuresh Nov 27, 2024 Maintainer

Uh oh!

ytooyama Nov 27, 2024 Author

Uh oh!

lalithsuresh Nov 27, 2024 Maintainer

Uh oh!

ytooyama Nov 27, 2024 Author

ytooyama
Nov 27, 2024

Replies: 4 comments 3 replies

ytooyama
Nov 27, 2024
Author

mihaibudiu
Nov 27, 2024
Maintainer

ytooyama
Nov 27, 2024
Author

lalithsuresh
Nov 27, 2024
Maintainer

ytooyama Nov 27, 2024
Author

lalithsuresh Nov 27, 2024
Maintainer

ytooyama Nov 27, 2024
Author