Joystick
Tutorials

How to Handle Uploads to S3 With Joystick

Learn how to configure and use Joystick's built-in uploader system to handle file uploads to Amazon S3, including validation, progress tracking, and form integration.

This tutorial was written by AI

The examples and explanations have been reviewed for accuracy. If you find an error, please let us know.

File uploads are a common requirement in web applications. Joystick provides a built-in uploader system that can handle uploads to multiple providers, including Amazon S3 and local storage. This tutorial will walk you through setting up and using Joystick's uploader system to handle file uploads to Amazon S3.

What You'll Learn

  • How to configure an uploader in your Joystick app
  • How to set up Amazon S3 credentials and permissions
  • How to validate file uploads (MIME types, file size)
  • How to track upload progress
  • How to create upload forms with file validation
  • How to handle upload responses and errors

Prerequisites

  • A Joystick app created and running
  • An Amazon S3 bucket set up
  • AWS credentials (Access Key ID and Secret Access Key)

Step 1: Configure Your S3 Bucket and Credentials

First, you'll need to set up your AWS credentials in your settings file. Add your S3 configuration to your settings.<env>.json file:

settings.development.json

{
  "config": {
    "databases": [
      {
        "provider": "mongodb",
        "users": true,
        "options": {}
      }
    ]
  },
  "global": {},
  "public": {},
  "private": {
    "aws": {
      "access_key_id": "YOUR_AWS_ACCESS_KEY_ID",
      "secret_access_key": "YOUR_AWS_SECRET_ACCESS_KEY"
    }
  }
}
Keep Credentials Secure

Always store AWS credentials in the private section of your settings file. Never commit real credentials to version control. Consider using environment variables in production.

Step 2: Define an Uploader

Create an uploader configuration in your index.server.js file. Uploaders are defined in the uploaders object passed to joystick.app():

index.server.js

import joystick from '@joystick.js/node';
import api from './api/index.js';

joystick.app({
  api,
  uploaders: {
    profile_photos: {
      providers: ['s3'],
      s3: {
        region: 'us-east-1',
        access_key_id: joystick?.settings?.private?.aws?.access_key_id,
        secret_access_key: joystick?.settings?.private?.aws?.secret_access_key,
        bucket: 'my-app-uploads',
        acl: 'public-read',
      },
      mime_types: ['image/png', 'image/jpg', 'image/jpeg', 'image/gif'],
      max_size_in_megabytes: 5,
      file_name: ({ input, file_name }) => {
        // Customize the file path/name
        const timestamp = new Date().getTime();
        return `profile-photos/${timestamp}-${file_name}`;
      },
      on_before_upload: ({ input, req, uploads }) => {
        // Validate or transform data before upload
        console.log('About to upload:', uploads.length, 'files');
      },
      on_after_upload: ({ input, req, uploads }) => {
        // Handle post-upload logic
        console.log('Upload completed:', uploads);
      },
    },
  },
  routes: {
    '/': (req = {}, res = {}) => {
      res.render('ui/pages/index/index.js', {
        layout: 'ui/layouts/app/index.js',
      });
    },
  },
});

Step 3: Create an Upload Form Component

Now let's create a component that handles file uploads. This component will include a form with file input and upload progress tracking:

ui/components/upload_form/index.js

import joystick, { upload } from '@joystick.js/ui';

const UploadForm = joystick.component({
  state: {
    uploading: false,
    upload_progress: 0,
    uploaded_files: [],
    error: null,
  },
  events: {
    'submit form': async (event = {}, instance = {}) => {
      event.preventDefault();
      
      const files = event.target.file.files;
      
      if (!files || files.length === 0) {
        instance.set_state({ error: 'Please select a file to upload.' });
        return;
      }
      
      instance.set_state({ 
        uploading: true, 
        upload_progress: 0, 
        error: null 
      });
      
      try {
        const response = await upload('profile_photos', {
          files: files,
          input: {
            user_id: instance.props?.user_id,
            description: event.target.description?.value || '',
          },
          on_progress: (percentage = 0, provider = '') => {
            instance.set_state({ upload_progress: percentage });
          },
        });
        
        instance.set_state({ 
          uploading: false,
          uploaded_files: response,
          upload_progress: 0,
        });
        
        // Reset the form
        event.target.reset();
        
      } catch (error) {
        instance.set_state({ 
          uploading: false,
          upload_progress: 0,
          error: error.message || 'Upload failed. Please try again.',
        });
      }
    },
    
    'change input[type="file"]': (event = {}, instance = {}) => {
      // Clear any previous errors when a new file is selected
      instance.set_state({ error: null });
    },
  },
  css: `
    .upload-form {
      max-width: 500px;
      margin: 0 auto;
      padding: 20px;
      border: 1px solid #ddd;
      border-radius: 8px;
    }
    
    .form-group {
      margin-bottom: 20px;
    }
    
    label {
      display: block;
      margin-bottom: 5px;
      font-weight: bold;
    }
    
    input[type="file"],
    input[type="text"],
    textarea {
      width: 100%;
      padding: 10px;
      border: 1px solid #ccc;
      border-radius: 4px;
      font-size: 16px;
    }
    
    button {
      background: #007bff;
      color: white;
      padding: 12px 24px;
      border: none;
      border-radius: 4px;
      cursor: pointer;
      font-size: 16px;
      width: 100%;
    }
    
    button:disabled {
      background: #6c757d;
      cursor: not-allowed;
    }
    
    .progress-bar {
      width: 100%;
      height: 20px;
      background: #f0f0f0;
      border-radius: 10px;
      overflow: hidden;
      margin: 10px 0;
    }
    
    .progress {
      height: 100%;
      background: #28a745;
      transition: width 0.3s ease;
    }
    
    .error {
      color: #dc3545;
      background: #f8d7da;
      padding: 10px;
      border-radius: 4px;
      margin-bottom: 15px;
    }
    
    .success {
      color: #155724;
      background: #d4edda;
      padding: 10px;
      border-radius: 4px;
      margin-bottom: 15px;
    }
    
    .uploaded-files {
      margin-top: 20px;
    }
    
    .file-item {
      display: flex;
      align-items: center;
      padding: 10px;
      background: #f8f9fa;
      border-radius: 4px;
      margin-bottom: 10px;
    }
    
    .file-item img {
      width: 50px;
      height: 50px;
      object-fit: cover;
      border-radius: 4px;
      margin-right: 10px;
    }
  `,
  render: ({ state, when }) => {
    return `
      <div class="upload-form">
        <h2>Upload Profile Photo</h2>
        
        ${when(state.error, `
          <div class="error">${state.error}</div>
        `)}
        
        ${when(state.uploaded_files.length > 0, `
          <div class="success">
            Upload completed successfully! ${state.uploaded_files.length} file(s) uploaded.
          </div>
        `)}
        
        <form>
          <div class="form-group">
            <label for="file">Choose File</label>
            <input 
              type="file" 
              name="file" 
              id="file" 
              accept="image/*"
              ${state.uploading ? 'disabled' : ''}
            />
            <small>Supported formats: PNG, JPG, JPEG, GIF. Max size: 5MB</small>
          </div>
          
          <div class="form-group">
            <label for="description">Description (Optional)</label>
            <input 
              type="text" 
              name="description" 
              id="description" 
              placeholder="Describe your photo..."
              ${state.uploading ? 'disabled' : ''}
            />
          </div>
          
          ${when(state.uploading, `
            <div class="progress-bar">
              <div class="progress" style="width: ${state.upload_progress}%;"></div>
            </div>
            <p>Uploading... ${state.upload_progress}%</p>
          `)}
          
          <button type="submit" ${state.uploading ? 'disabled' : ''}>
            ${state.uploading ? 'Uploading...' : 'Upload Photo'}
          </button>
        </form>
        
        ${when(state.uploaded_files.length > 0, `
          <div class="uploaded-files">
            <h3>Uploaded Files</h3>
            ${state.uploaded_files.map(file => `
              <div class="file-item">
                <img src="${file.url}" alt="Uploaded file" />
                <div>
                  <strong>${file.original_name}</strong><br>
                  <small>${file.url}</small>
                </div>
              </div>
            `).join('')}
          </div>
        `)}
      </div>
    `;
  },
});

export default UploadForm;

Step 4: Use the Upload Component

Now let's use our upload component in a page:

ui/pages/profile/index.js

import joystick from '@joystick.js/ui';
import UploadForm from '../../components/upload_form/index.js';

const Profile = joystick.component({
  data: async (api = {}, req = {}) => {
    // You could fetch user data here if needed
    return {
      user: req?.context?.user || null,
    };
  },
  render: ({ data, component }) => {
    return `
      <div class="profile-page">
        <h1>Profile Settings</h1>
        
        ${component(UploadForm, {
          user_id: data?.user?._id,
        })}
      </div>
    `;
  },
});

export default Profile;

Step 5: Add Advanced Upload Features

Let's enhance our uploader with additional features like multiple file support and custom validation:

index.server.js (Enhanced Uploader)

import joystick from '@joystick.js/node';

joystick.app({
  uploaders: {
    document_uploads: {
      providers: ['local', 's3'], // Support both local and S3
      local: {
        path: 'uploads/documents',
      },
      s3: {
        region: 'us-east-1',
        access_key_id: joystick?.settings?.private?.aws?.access_key_id,
        secret_access_key: joystick?.settings?.private?.aws?.secret_access_key,
        bucket: 'my-app-documents',
        acl: 'private', // Private files
      },
      mime_types: [
        'application/pdf',
        'application/msword',
        'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
        'text/plain',
      ],
      max_size_in_megabytes: (input) => {
        // Dynamic file size based on user type
        return input?.user_type === 'premium' ? 50 : 10;
      },
      file_name: ({ input, file_name }) => {
        const user_id = input?.user_id || 'anonymous';
        const timestamp = new Date().getTime();
        const sanitized_name = file_name.replace(/[^a-zA-Z0-9.-]/g, '_');
        return `documents/${user_id}/${timestamp}-${sanitized_name}`;
      },
      on_before_upload: ({ input, req, uploads }) => {
        // Log upload attempt
        console.log(`User ${input?.user_id} uploading ${uploads.length} files`);
        
        // You could add additional validation here
        if (!input?.user_id) {
          throw new Error('User ID is required for document uploads');
        }
      },
      on_after_upload: async ({ input, req, uploads }) => {
        // Save upload records to database
        for (const upload of uploads) {
          await process.databases.mongodb.collection('uploads').insertOne({
            _id: joystick.id(),
            user_id: input?.user_id,
            file_name: upload.original_name,
            file_url: upload.url,
            file_size: upload.size,
            mime_type: upload.mime_type,
            uploaded_at: new Date().toISOString(),
          });
        }
        
        console.log('Upload records saved to database');
      },
    },
  },
  routes: {
    // Your routes here
  },
});

Step 6: Handle Upload Errors and Validation

Create a more robust upload component that handles various error scenarios:

ui/components/document_uploader/index.js

import joystick, { upload } from '@joystick.js/ui';

const DocumentUploader = joystick.component({
  state: {
    uploading: false,
    upload_progress: 0,
    uploaded_files: [],
    errors: [],
    selected_files: [],
  },
  methods: {
    validate_files: (files = [], instance = {}) => {
      const errors = [];
      const max_size = 10 * 1024 * 1024; // 10MB in bytes
      const allowed_types = [
        'application/pdf',
        'application/msword',
        'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
        'text/plain',
      ];
      
      Array.from(files).forEach((file, index) => {
        if (file.size > max_size) {
          errors.push(`File ${index + 1}: Size exceeds 10MB limit`);
        }
        
        if (!allowed_types.includes(file.type)) {
          errors.push(`File ${index + 1}: Unsupported file type (${file.type})`);
        }
      });
      
      return errors;
    },
  },
  events: {
    'change input[type="file"]': (event = {}, instance = {}) => {
      const files = event.target.files;
      const errors = instance.methods.validate_files(files);
      
      instance.set_state({ 
        selected_files: Array.from(files),
        errors: errors,
      });
    },
    
    'submit form': async (event = {}, instance = {}) => {
      event.preventDefault();
      
      const files = event.target.file.files;
      const errors = instance.methods.validate_files(files);
      
      if (errors.length > 0) {
        instance.set_state({ errors });
        return;
      }
      
      if (!files || files.length === 0) {
        instance.set_state({ errors: ['Please select at least one file to upload.'] });
        return;
      }
      
      instance.set_state({ 
        uploading: true, 
        upload_progress: 0, 
        errors: [] 
      });
      
      try {
        const response = await upload('document_uploads', {
          files: files,
          input: {
            user_id: instance.props?.user_id,
            user_type: instance.props?.user_type || 'standard',
            category: event.target.category?.value || 'general',
          },
          on_progress: (percentage = 0, provider = '') => {
            instance.set_state({ upload_progress: percentage });
          },
        });
        
        instance.set_state({ 
          uploading: false,
          uploaded_files: [...instance.state.uploaded_files, ...response],
          upload_progress: 0,
          selected_files: [],
        });
        
        // Reset the form
        event.target.reset();
        
      } catch (error) {
        let error_message = 'Upload failed. Please try again.';
        
        if (error.message.includes('File size')) {
          error_message = 'One or more files exceed the size limit.';
        } else if (error.message.includes('MIME type')) {
          error_message = 'One or more files have an unsupported format.';
        }
        
        instance.set_state({ 
          uploading: false,
          upload_progress: 0,
          errors: [error_message],
        });
      }
    },
  },
  css: `
    .document-uploader {
      max-width: 600px;
      margin: 0 auto;
      padding: 20px;
    }
    
    .errors {
      background: #f8d7da;
      color: #721c24;
      padding: 15px;
      border-radius: 4px;
      margin-bottom: 20px;
    }
    
    .errors ul {
      margin: 0;
      padding-left: 20px;
    }
    
    .file-preview {
      background: #f8f9fa;
      padding: 15px;
      border-radius: 4px;
      margin: 15px 0;
    }
    
    .file-preview h4 {
      margin: 0 0 10px 0;
    }
    
    .file-list {
      list-style: none;
      padding: 0;
    }
    
    .file-list li {
      padding: 5px 0;
      border-bottom: 1px solid #eee;
    }
    
    .uploaded-documents {
      margin-top: 30px;
      padding-top: 20px;
      border-top: 1px solid #ddd;
    }
    
    .document-item {
      display: flex;
      justify-content: space-between;
      align-items: center;
      padding: 15px;
      background: #f8f9fa;
      border-radius: 4px;
      margin-bottom: 10px;
    }
    
    .document-info h4 {
      margin: 0 0 5px 0;
    }
    
    .document-info small {
      color: #6c757d;
    }
    
    .download-link {
      background: #28a745;
      color: white;
      padding: 8px 16px;
      text-decoration: none;
      border-radius: 4px;
      font-size: 14px;
    }
  `,
  render: ({ state, when, each }) => {
    return `
      <div class="document-uploader">
        <h2>Upload Documents</h2>
        
        ${when(state.errors.length > 0, `
          <div class="errors">
            <strong>Please fix the following errors:</strong>
            <ul>
              ${each(state.errors, (error) => `<li>${error}</li>`)}
            </ul>
          </div>
        `)}
        
        <form>
          <div class="form-group">
            <label for="file">Choose Documents</label>
            <input 
              type="file" 
              name="file" 
              id="file" 
              multiple
              accept=".pdf,.doc,.docx,.txt"
              ${state.uploading ? 'disabled' : ''}
            />
            <small>Supported formats: PDF, DOC, DOCX, TXT. Max size: 10MB per file.</small>
          </div>
          
          ${when(state.selected_files.length > 0, `
            <div class="file-preview">
              <h4>Selected Files:</h4>
              <ul class="file-list">
                ${each(state.selected_files, (file) => `
                  <li>${file.name} (${Math.round(file.size / 1024)}KB)</li>
                `)}
              </ul>
            </div>
          `)}
          
          <div class="form-group">
            <label for="category">Category</label>
            <select name="category" id="category" ${state.uploading ? 'disabled' : ''}>
              <option value="general">General</option>
              <option value="contracts">Contracts</option>
              <option value="invoices">Invoices</option>
              <option value="reports">Reports</option>
            </select>
          </div>
          
          ${when(state.uploading, `
            <div class="progress-bar">
              <div class="progress" style="width: ${state.upload_progress}%;"></div>
            </div>
            <p>Uploading... ${state.upload_progress}%</p>
          `)}
          
          <button type="submit" ${state.uploading || state.selected_files.length === 0 ? 'disabled' : ''}>
            ${state.uploading ? 'Uploading...' : `Upload ${state.selected_files.length} File(s)`}
          </button>
        </form>
        
        ${when(state.uploaded_files.length > 0, `
          <div class="uploaded-documents">
            <h3>Uploaded Documents</h3>
            ${each(state.uploaded_files, (file) => `
              <div class="document-item">
                <div class="document-info">
                  <h4>${file.original_name}</h4>
                  <small>Uploaded: ${new Date().toLocaleDateString()}</small>
                </div>
                <a href="${file.url}" class="download-link" target="_blank">Download</a>
              </div>
            `)}
          </div>
        `)}
      </div>
    `;
  },
});

export default DocumentUploader;

Step 7: Testing Your Upload System

Create a simple test to verify your upload system works correctly:

tests/uploaders/profile_photos.test.js

import test from '@joystick.js/test';

test.that('profile photo uploader works correctly', async (assert = {}) => {
  const user = await test.accounts.signup({
    email_address: 'test@example.com',
    password: 'password123',
  });

  // Create a mock image file
  const mock_file = test.utils.create_file(1024, 'test-photo.jpg', 'image/jpeg');

  const response = await test.uploaders.upload('profile_photos', {
    user,
    files: [mock_file],
    input: {
      user_id: user._id,
      description: 'Test profile photo',
    },
  });

  assert.is(response.length, 1);
  assert.is(response[0].original_name, 'test-photo.jpg');
  assert.is(typeof response[0].url, 'string');
});

Best Practices

  1. Security: Always validate file types and sizes on both client and server
  2. Error Handling: Provide clear error messages for different failure scenarios
  3. Progress Tracking: Show upload progress for better user experience
  4. File Organization: Use meaningful file paths and naming conventions
  5. Cleanup: Consider implementing file cleanup for failed uploads
  6. Testing: Test your upload system with various file types and sizes

Troubleshooting

Upload fails with "Access Denied": Check your S3 bucket permissions and AWS credentials.

Files are too large: Verify your max_size_in_megabytes setting and S3 upload limits.

Wrong file type uploaded: Ensure your mime_types array includes all allowed formats.

Progress not updating: Make sure you're calling instance.set_state() in the on_progress callback.

Next Steps

  • Implement image resizing before upload
  • Add drag-and-drop file upload functionality
  • Create a file management system with delete capabilities
  • Add upload queues for large files
  • Implement file sharing and permissions

You now have a complete file upload system that can handle uploads to Amazon S3 with validation, progress tracking, and error handling!