Skip to content

Conversation

@shreyashidabral
Copy link

@shreyashidabral shreyashidabral commented Oct 6, 2023

Fixes: T348188

Changes Made

  • Check if there is direct pdf download option available for books/newsletters/magazines.
  • If yes, enable download and automate the upload without parsing each image separately.

Files Updated

  • utils/helper.js - Implement a check to find if direct pdf download option is available for the pdl resource, and add it to metadata if available.
  • bull/pdl-queue/consumer.js - Create a new function to download the pdf and automate its upload without parsing each image separately. After this I added suitable content type headers by adding a validation to check for pdf or zip.

const IAuri = `http://s3.us.archive.org/${bucketTitle}/${bucketTitle}_images.zip`;
metadata = _.omit(metadata, "coverImage");
let headers = setHeaders(metadata, byteLength, metadata.title);
let headers = setHeaders(metadata, byteLength, metadata.title, zip);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. "zip" should be in string.
  2. I think it's better you directly send the content type so that you don't need to do another check in the setHeaders() function.

const requestURI = request(job.data.uri);
const bucketTitle = metadata.IAIdentifier;
const IAuri = `http://s3.us.archive.org/${bucketTitle}/${bucketTitle}.pdf`;
let headers = setHeaders(metadata, responseSize, metadata.title, pdf);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. "pdf" should be in string.
  2. I think it's better you directly send the content type so that you don't need to do another check in the setHeaders() function.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made the requested changes

@coderwassananmol
Copy link
Owner

@shreyashidabral This PR is failing for me with the below error:
Screenshot 2023-10-09 at 1 25 21 AM

I tried uploading this book from PDL.

Copy link
Owner

@coderwassananmol coderwassananmol left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2023-10-10 at 1 18 12 AM @shreyashidabral The new changes are failing with this error. Please fix and resubmit.

@coderwassananmol
Copy link
Owner

Closing this since #224 is being worked upon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants