1

I am having a terrible time trying to figure out why duplicate files are being uploaded. Here's what's happening:

  1. We upload photos of varying size (i.e. 75 width, 300 width, 1920 width, etc).
  2. The upload succeeds and I can see in the logging that no photos are created twice. The photo upload is run in sequence, but EVERY TIME there are two 300width records in S3. It ONLY happens on photos of that size, every time.

Even if I run some code that checks to see if the object exists already, there are duplicates created for that specific width.

Here is the S3 class upload code:

def self.upload(file, bucket, object_path)
    # Build and setup path to object
    unless bucket.object(object_path).exists?
      obj = bucket.object(object_path)
      # Upload file to object path
      obj.upload_file(file.path, acl: 'public-read')
      # # Return S3 public url of cached image
      obj.public_url({virtual_host: true}).gsub("REDACTED.com", "cdn.REDACTED.com").gsub("http://", "https://")
    end
  end

The duplicates are literally the exact same. It's not a UI glitch, I can see them both in S3.bucket.objects.to_a

Summary: duplicate 300w uploaded everytime, even though all the sizes are uploaded together, with one particular file being duplicated consistently every time.

Anybody have any ideas?

EDIT

Here's the code where the caches are being iterated and uploaded:

initial_cache_sizes = [
  {width: 75, height: 50},
  # Width 300
  {width: 300, height: 200},
  # Width 830
  {width: 830, height: 553},
]
caches = initial_cache_sizes.map {|cache| create_cache(cache)}

# create an object to be uploaded
def create_cache(attributes)
  Cache.new(self, attributes).build
end

class Cache
  # Tableless Model | Extending methods
  extend ActiveModel::Naming
  # Tableless Model | Setting attributes
  attr_accessor :cacheable, :size, :width, :height, :path, :disclaimer, :aspect_ratio, :vendor_watermark, :company_watermark, :file
  # Tableless Model | Initialize based on PlanOcore
  def initialize(cacheable, attributes = {})
    # Build model from attributes
    attributes.each do |name, value|
      # Save value to model
      send("#{name}=", value)
    end
    # Save photo or floor_plan to cacheable
    send("cacheable=", cacheable)
    # Save file to cache
    send("file=", MiniMagick::Image.open(cacheable.path))
  end
  # Build cache file
  def build
    begin
      # Resize if width and height are given.
      resize if width && height
      # Resize if a size was given.
      crop(aspect_ratio) if aspect_ratio
      # Add vendor watermark if asked for and available.
      add_watermark(cacheable.vendor_watermark, "south-east") if vendor_watermark
      # Add company watermark if asked for and available.
      add_watermark(cacheable.company_watermark, "south-west") if company_watermark
      # Add disclaimer is asked for.
      add_disclaimer if disclaimer
      # Upload to S3
      upload
    ensure
      unlink # Delete tempfile
    end
    response_format # Return cache.
  end

def upload
  # Build S3 path.
  s3_path = "#{cacheable.s3_path}/#{file_name}"
  # Upload file to S3
  send("path=", S3.upload(file, cacheable.bucket, s3_path))
end

# S3.upload code outlined above

This is the output of logging the caches object from caches = initial_cache_sizes.map {|cache| create_cache(cache)}

{:size=>5041, :width=>75, :height=>50, :path=>"REDACTED", :key=>"75w50h", :disclaimer=>nil, :vendor_watermark=>nil, :company_watermark=>nil}
{:size=>37317, :width=>300, :height=>200, :path=>"REDACTED", :key=>"300w200h", :disclaimer=>nil, :vendor_watermark=>nil, :company_watermark=>nil}
{:size=>186293, :width=>830, :height=>553, :path=>"REDACTED", :key=>"830w553h", :disclaimer=>nil, :vendor_watermark=>nil, :company_watermark=>nil}
{:size=>264617, :width=>1024, :height=>682, :path=>"REDACTED", :key=>"1024w682h", :disclaimer=>nil, :vendor_watermark=>nil, :company_watermark=>nil}
{:size=>787140, :width=>1920, :height=>1280, :path=>"REDACTED", :key=>"1920w1280h", :disclaimer=>nil, :vendor_watermark=>nil, :company_watermark=>nil}

As you can see, only ONE 300 width file, yet there are two present in S3.

6
  • Have you ruled out that it's caused by your code which calls upload? E.g. are you sure you're calling it the correct amount of times, once for each image size? Maybe you can share that code for verification. Even still, I guess the exists? check should work, so not really sure. Commented Apr 1, 2020 at 19:02
  • I added all the upload code! Commented Apr 1, 2020 at 19:10
  • ****ing wierd, I have no clue Commented Apr 1, 2020 at 20:21
  • What do you mean by "duplicated"? Wouldn't they have different filenames (Keys)? Commented Apr 2, 2020 at 7:49
  • No, it's the exact same filename. Same size, same modified_at, different etag. Commented Apr 2, 2020 at 15:24

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.