-
Notifications
You must be signed in to change notification settings - Fork 501
Add S3 bucket #816
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Add S3 bucket #816
Changes from all commits
Commits
Show all changes
95 commits
Select commit
Hold shift + click to select a range
0f9b372
add tenancy crud
fomalhautb a9ceab2
remove old config
fomalhautb 6390832
fix validateRedirectUrl
fomalhautb 1f02532
fix tenancy config
fomalhautb 588019d
fix
fomalhautb 86466a9
better error handling
fomalhautb 3d38936
fix
fomalhautb 506ff7d
fix
fomalhautb 5d7bcc0
fix
fomalhautb 3626c5c
fix
fomalhautb 76f593a
fix
fomalhautb 0b00d8c
remove environment-config
fomalhautb 62b7775
Update apps/backend/src/app/api/latest/integrations/neon/oauth-provid…
fomalhautb 0bcafcd
Update apps/backend/src/app/api/latest/auth/otp/send-sign-in-code/rou…
fomalhautb 15f73de
Update apps/backend/prisma/seed.ts
fomalhautb 8c18c7f
Merge branch 'dev' into remove-old-config
fomalhautb 073ab15
fix
fomalhautb bc069b0
fix tests
fomalhautb 9ceb0ac
added config override crud
fomalhautb 1afbf06
rename
fomalhautb c18cae3
Refactor updateConfigOverrides to handle legacy config structure and …
fomalhautb 0387792
fix bugs
fomalhautb 6458c54
add use config
fomalhautb 408e387
added todos
fomalhautb 41d8a4c
tests
fomalhautb e2cb2ab
tests
fomalhautb dabc619
tests
fomalhautb 32afeb2
more tests
fomalhautb 4e25c86
init
fomalhautb 13bb68e
s3 image
fomalhautb 0a3e088
fix size
fomalhautb 1b9d1b0
updated image profile url
fomalhautb 3a6c548
Merge branch 'dev' into s3
fomalhautb 431a3a4
env vars
fomalhautb f61670a
fix package.json
fomalhautb cb7d262
fix tests
fomalhautb 7a648b2
fix tests
fomalhautb 1fd77f6
add to emulator
fomalhautb f5d6656
Merge branch 'dev' into s3
fomalhautb b2722a4
dynamic import
fomalhautb e4a6b04
Merge branch 's3' of github.com:stackframe-projects/stack into s3
fomalhautb 73d90ca
Merge branch 'dev' into s3
fomalhautb c883965
make s3 optional
fomalhautb e929f74
docs
fomalhautb 48debe3
remove test route
fomalhautb 814b66d
Update apps/backend/.env
fomalhautb 517b4dd
improve error handling
fomalhautb 0d4fd41
Merge branch 's3' of github.com:stackframe-projects/stack into s3
fomalhautb 3a2def7
comments
fomalhautb c856d00
Update S3 configuration checks and regex for base64 image validation
fomalhautb 64b34b1
fix import error
fomalhautb 3369c3a
fix import
fomalhautb af5fbd0
fix syntax error
fomalhautb 6d83076
fix types
fomalhautb 4ed5fe9
Merge branch 'config-json-crud' into s3
fomalhautb 6172952
fix
fomalhautb d2413f3
fix types
fomalhautb 7f78755
fix types
fomalhautb 405a63f
Merge branch 'dev' into remove-old-config
fomalhautb 1904feb
Merge branch 'dev' into remove-old-config
fomalhautb 079df58
fix types
fomalhautb 9fa42c7
Improve error handling for missing provider type in OAuth configuration
fomalhautb 577eaf3
fix tests
fomalhautb ae04ae2
fix tests
fomalhautb 40a6c7a
Merge branch 'remove-old-config' into project-config-to-json
fomalhautb 60ced05
Merge branch 'dev' into project-config-to-json
fomalhautb 5a05e01
fix
fomalhautb 9e5a049
Update apps/e2e/tests/backend/endpoints/api/v1/internal/config-overri…
fomalhautb c76d55f
fix
fomalhautb 40db4e2
Merge branch 'project-config-to-json' of github.com:stackframe-projec…
fomalhautb 7862a3e
Refactor config overrides CRUD handlers by simplifying paramsSchema a…
fomalhautb ee330cb
Refactor config CRUD handlers to standardize naming conventions for c…
fomalhautb b3b581c
fix tests
fomalhautb 5d45170
Merge branch 'dev' into project-config-to-json
fomalhautb 1d52910
fix
fomalhautb 677de8a
fix
fomalhautb 44eecf6
Merge branch 'dev' into project-config-to-json
fomalhautb db50e41
fix
fomalhautb 48bb875
add tests
fomalhautb eb5a173
Merge branch 'dev' into project-config-to-json
fomalhautb 8e581f6
remove logging
fomalhautb dee426f
remove word
fomalhautb 3d20abc
rename endpoints
fomalhautb 31d8c37
Merge branch 'project-config-to-json' into s3
fomalhautb 52cfaf8
fix
fomalhautb a205566
removed unused
fomalhautb a6c69bb
Merge branch 'dev' into s3
fomalhautb f103ce8
Merge branch 's3' of github.com:stackframe-projects/stack into s3
fomalhautb c324aa2
remove unused
fomalhautb c480271
fix import
fomalhautb 95db583
8120 -> 8121
fomalhautb f7b1d20
fix test
fomalhautb 6cfc7b7
improve auto-migration test
fomalhautb d518eb0
refactor auto-migration test to simplify success count validation
fomalhautb 901f0b5
fix
fomalhautb File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| export class ImageProcessingError extends Error { | ||
| constructor(message: string) { | ||
| super(message); | ||
| this.name = 'ImageProcessingError'; | ||
| } | ||
| } | ||
|
|
||
| export async function parseBase64Image(input: string, options: { | ||
| maxBytes?: number, | ||
| maxWidth?: number, | ||
| maxHeight?: number, | ||
| allowTypes?: string[], | ||
| } = { | ||
| maxBytes: 1024 * 300, | ||
| maxWidth: 4096, | ||
| maxHeight: 4096, | ||
| allowTypes: ['image/jpeg', 'image/png', 'image/webp'], | ||
| }) { | ||
| // Remove data URL prefix if present (e.g., "data:image/jpeg;base64,") | ||
| const base64Data = input.replace(/^data:image\/[a-zA-Z0-9]+;base64,/, ''); | ||
|
|
||
| // check the size before and after the base64 conversion | ||
| if (base64Data.length > options.maxBytes!) { | ||
| throw new ImageProcessingError(`Image size (${base64Data.length} bytes) exceeds maximum allowed size (${options.maxBytes} bytes)`); | ||
| } | ||
fomalhautb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| // Convert base64 to buffer | ||
| let imageBuffer: Buffer; | ||
| try { | ||
| imageBuffer = Buffer.from(base64Data, 'base64'); | ||
| } catch (error) { | ||
| throw new ImageProcessingError('Invalid base64 image data'); | ||
| } | ||
|
|
||
| // Check file size | ||
| if (options.maxBytes && imageBuffer.length > options.maxBytes) { | ||
| throw new ImageProcessingError(`Image size (${imageBuffer.length} bytes) exceeds maximum allowed size (${options.maxBytes} bytes)`); | ||
| } | ||
|
|
||
| // Dynamically import sharp | ||
| const sharp = (await import('sharp')).default; | ||
|
|
||
| // Use Sharp to load image and get metadata | ||
| let sharpImage: any; | ||
| let metadata: any; | ||
fomalhautb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| try { | ||
| sharpImage = sharp(imageBuffer); | ||
| metadata = await sharpImage.metadata(); | ||
| } catch (error) { | ||
| throw new ImageProcessingError('Invalid image format or corrupted image data'); | ||
| } | ||
|
|
||
| // Validate image format | ||
| if (!metadata.format) { | ||
| throw new ImageProcessingError('Unable to determine image format'); | ||
| } | ||
|
|
||
| const mimeType = `image/${metadata.format}`; | ||
| if (options.allowTypes && !options.allowTypes.includes(mimeType)) { | ||
| throw new ImageProcessingError(`Image type ${mimeType} is not allowed. Allowed types: ${options.allowTypes.join(', ')}`); | ||
| } | ||
|
|
||
| if (!metadata.width || !metadata.height) { | ||
| throw new ImageProcessingError('Unable to determine image dimensions'); | ||
| } | ||
|
|
||
| if (options.maxWidth && metadata.width > options.maxWidth) { | ||
| throw new ImageProcessingError(`Image width (${metadata.width}px) exceeds maximum allowed width (${options.maxWidth}px)`); | ||
| } | ||
|
|
||
| if (options.maxHeight && metadata.height > options.maxHeight) { | ||
| throw new ImageProcessingError(`Image height (${metadata.height}px) exceeds maximum allowed height (${options.maxHeight}px)`); | ||
| } | ||
|
|
||
| // Return the validated image data and metadata | ||
| return { | ||
| buffer: imageBuffer, | ||
| metadata: { | ||
| format: metadata.format, | ||
| mimeType, | ||
| width: metadata.width, | ||
| height: metadata.height, | ||
| size: imageBuffer.length, | ||
| channels: metadata.channels, | ||
| density: metadata.density, | ||
| hasProfile: metadata.hasProfile, | ||
| hasAlpha: metadata.hasAlpha, | ||
| }, | ||
| sharp: sharpImage, | ||
| }; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| import { PutObjectCommand, S3Client } from "@aws-sdk/client-s3"; | ||
| import { getEnvVariable } from "@stackframe/stack-shared/dist/utils/env"; | ||
| import { StackAssertionError, StatusError } from "@stackframe/stack-shared/dist/utils/errors"; | ||
| import { ImageProcessingError, parseBase64Image } from "./lib/images"; | ||
|
|
||
| const S3_REGION = getEnvVariable("STACK_S3_REGION", ""); | ||
| const S3_ENDPOINT = getEnvVariable("STACK_S3_ENDPOINT", ""); | ||
| const S3_BUCKET = getEnvVariable("STACK_S3_BUCKET", ""); | ||
| const S3_ACCESS_KEY_ID = getEnvVariable("STACK_S3_ACCESS_KEY_ID", ""); | ||
| const S3_SECRET_ACCESS_KEY = getEnvVariable("STACK_S3_SECRET_ACCESS_KEY", ""); | ||
|
|
||
| const HAS_S3 = !!S3_REGION && !!S3_ENDPOINT && !!S3_BUCKET && !!S3_ACCESS_KEY_ID && !!S3_SECRET_ACCESS_KEY; | ||
|
|
||
| if (!HAS_S3) { | ||
| console.warn("S3 bucket is not configured. File upload features will not be available."); | ||
| } | ||
|
|
||
| const s3Client = HAS_S3 ? new S3Client({ | ||
| region: S3_REGION, | ||
| endpoint: S3_ENDPOINT, | ||
fomalhautb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| forcePathStyle: true, | ||
| credentials: { | ||
| accessKeyId: S3_ACCESS_KEY_ID, | ||
| secretAccessKey: S3_SECRET_ACCESS_KEY, | ||
| }, | ||
| }) : undefined; | ||
|
|
||
| export function getS3PublicUrl(key: string): string { | ||
| return `${S3_ENDPOINT}/${S3_BUCKET}/${key}`; | ||
| } | ||
fomalhautb marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| async function uploadBase64Image({ | ||
| input, | ||
| maxBytes = 1024 * 300, | ||
| folderName, | ||
| }: { | ||
| input: string, | ||
| maxBytes?: number, | ||
| folderName: string, | ||
| }) { | ||
| if (!s3Client) { | ||
| throw new StackAssertionError("S3 is not configured"); | ||
| } | ||
|
|
||
| let buffer: Buffer; | ||
| let format: string; | ||
| try { | ||
| const result = await parseBase64Image(input, { maxBytes }); | ||
| buffer = result.buffer; | ||
| format = result.metadata.format; | ||
| } catch (error) { | ||
| if (error instanceof ImageProcessingError) { | ||
| throw new StatusError(StatusError.BadRequest, error.message); | ||
| } | ||
| throw error; | ||
| } | ||
|
|
||
| const key = `${folderName}/${crypto.randomUUID()}.${format}`; | ||
|
|
||
| const command = new PutObjectCommand({ | ||
| Bucket: S3_BUCKET, | ||
| Key: key, | ||
| Body: buffer, | ||
| }); | ||
|
|
||
| await s3Client.send(command); | ||
|
|
||
| return { | ||
| key, | ||
| url: getS3PublicUrl(key), | ||
| }; | ||
| } | ||
|
|
||
| export function checkImageString(input: string) { | ||
| return { | ||
| isBase64Image: /^data:image\/[a-zA-Z0-9]+;base64,/.test(input), | ||
| isUrl: /^https?:\/\//.test(input), | ||
| }; | ||
| } | ||
|
|
||
| export async function uploadAndGetUrl( | ||
| input: string | null | undefined, | ||
| folderName: 'user-profile-images' | 'team-profile-images' | 'team-member-profile-images' | ||
| ) { | ||
| if (input) { | ||
| const checkResult = checkImageString(input); | ||
| if (checkResult.isBase64Image) { | ||
| const { url } = await uploadBase64Image({ input, folderName }); | ||
| return url; | ||
| } else if (checkResult.isUrl) { | ||
| return input; | ||
| } else { | ||
| throw new StatusError(StatusError.BadRequest, "Invalid profile image URL"); | ||
| } | ||
|
|
||
| } else if (input === null) { | ||
| return null; | ||
| } else { | ||
| return undefined; | ||
| } | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.