By Rosan International | | Cloud Storage

 

OK, this is an embarrassingly simple task, but one for which I couldn’t  get a straight answer from either Stackoverflow or ChatGPT… So here it is for your great convenience! Let us assume that you have a Firebase Storage bucket with a bunch of files in different folders that you want to download and zip into a single file for offline use. In my case, I want to get videos from Firebase storage and use them for a video detection machine learning training task in Google Colab.

Generate Private Access Key

First you need to save an access key locally to authenticate your API request from Python.

  • Access Firebase Console: Log in to the Firebase Console using your Google account.
  • Select Your Project: Choose the Firebase project for which you want to create a service account.
  • Go to Project Settings: Click on the gear icon (settings) located in the top left corner to access your project’s settings.
  • Navigate to the Service Accounts Tab: In the left sidebar, select the “Service accounts” tab.
  • Generate New Private Key: Scroll down to the “Firebase Admin SDK” section and click on the “Generate new private key” button.
  • Secure the Private Key: Keep the private key file secure, as it provides access to your Firebase project. Store it in a safe location, and avoid sharing it publicly.

Launch Colab

  • Open Colab: Launch your preferred web browser on your computer. In the address bar of your web browser, type “colab.research.google.com” and press Enter. You will be directed to the Google Colab homepage. If you’re not already signed in with your Google account, click the “Sign in” button in the top right corner and sign in with your Google credentials.
  • Create a New Notebook or Open an Existing One: After signing in, you’ll see the Colab dashboard. Here, you can create a new Colab notebook by clicking on “New Notebook” or open an existing notebook from your Google Drive by clicking on “File” and then selecting “Open Notebook.”
  • Upload private access key: Let’s just “no code it” and click the Files icon to open the “Files explorer” pane, then click the upload icon and select the json file with the private access key from the “File Upload” dialog window.

Setup Environment

Next create a code cell in Colab to setup and test your private access key.

# Import libraries

import firebase_admin
from firebase_admin import credentials
from firebase_admin import firestore
# Get credentials
cred = credentials.Certificate(‘/content/serviceAccountKey.json’)
firebase_admin.initialize_app(cred)
# Get a reference to the storage service
from firebase_admin import storage
storage_client = storage
# Reference the specific Firebase Storage bucket you want to access
bucket = storage_client.bucket(‘epicam-3311a.appspot.com’)

# List files in a specific folder
folder_name = ‘Video_1’
blobs = bucket.list_blobs(prefix=folder_name)
for blob in blobs:
if blob.name.endswith(‘.mp4’):
print(blob.name)

Video_1/1692996142417.mp4 
Video_1/1692996190814.mp4 
Video_1/1692996248307.mp4 
Video_1/1692996931992.mp4 
Video_1/1692996957889.mp4 
Video_1/1692997009409.mp4

Zip it!

Next we use the os and zipfile libraries to recursively compress all files and folders within a specified directory into a single ZIP file. The code below iterates through the contents of the source directory, adds each file to the ZIP archive, and maintains the directory structure within the archive. The resulting ZIP file, named “compressed_folders.zip” in this example, will contain all the files and subdirectories from the source directory, preserving their relative paths.

import zipfile
import os

# List of folder names to be compressed

folder_names = [‘Video_1’, ‘Video_2’, ‘Video_3’]
# Name of the output ZIP file
zip_file_name = ‘compressed_folders.zip’

# Temporary directory to store downloaded files

temp_dir = ‘/tmp/compressed_folders’
os.makedirs(temp_dir, exist_ok=True)
# Create a new zip file
with zipfile.ZipFile(zip_file_name, ‘w’, zipfile.ZIP_DEFLATED) as zipf:
    for folder_name in folder_names:
        blobs = bucket.list_blobs(prefix=folder_name)
        for blob in blobs:
            if blob.name.endswith(‘.mp4’):
                # Download the file to a temporary location
                tmp_file_name = os.path.join(temp_dir, os.path.basename(blob.name))
                blob.download_to_filename(tmp_file_name)
                # Add the downloaded file to the zip
                zipf.write(tmp_file_name, os.path.basename(tmp_file_name))
# Remove the temporary directory and its contents
for file_name in os.listdir(temp_dir):
    file_path = os.path.join(temp_dir, file_name)
    os.remove(file_path)
os.rmdir(temp_dir)
print(f’Compressed folders into {zip_file_name}’)
Compressed folders into compressed_folders.zip

And there you have it! All the app-friendly goodness of Firebase is now linked to your sandbox ML environment in Colab. Now go build something awesome!

About Rosan International

ROSAN is a technology company specialized in the development of Data Science and Artificial Intelligence solutions with the aim of solving the most challenging global projects. Contact us to discover how we can help you gain valuable insights from your data and optimize your processes.