Need Security Automation? Try Open-Source Shuffler

by

Need Security Automation? Try Open-Source Shuffler

In today’s complex threat landscape, security teams are constantly overwhelmed with alerts and incidents. Shuffler is an open-source security automation and orchestration platform designed to alleviate this burden. It empowers analysts to automate repetitive tasks, correlate security data from various sources, and respond to threats more effectively. This comprehensive guide explores Shuffler’s capabilities, installation, usage, and best practices.

Overview of Shuffler

Top view of composition of blackboard with written phrase STOP KILLING US under mask against black background
Top view of composition of blackboard with written phrase STOP KILLING US under mask against black background

Shuffler is more than just a security tool; it’s a modular, extensible platform for building custom security workflows. At its core, it provides a visual interface for designing and executing automated incident response processes. Imagine a scenario where a phishing email is detected. Instead of manually investigating the email, identifying affected users, and isolating compromised systems, you can create a Shuffler workflow that automatically performs these actions. Shuffler connects to various security tools and data sources, allowing analysts to centralize their investigations and responses. The ingenious aspect of Shuffler lies in its flexibility; you can tailor it to your specific environment and security needs. It acts as a security orchestration, automation, and response (SOAR) platform, but without the heavy price tag often associated with commercial solutions. It’s designed for analysts, by analysts, providing an intuitive interface that encourages collaboration and knowledge sharing. Shuffler’s value resides in how it helps teams become more proactive in threat hunting, incident response, and security operations in general.

Installation of Shuffler

Child in distress on checkered chair with adult leaning over ominously inside a dimly lit room.
Child in distress on checkered chair with adult leaning over ominously inside a dimly lit room.

Shuffler offers several installation options, including Docker Compose, Kubernetes, and manual installation. The Docker Compose method is the simplest and most recommended for getting started. Here’s a step-by-step guide:

  1. Prerequisites: Ensure you have Docker and Docker Compose installed on your system. You can download them from the official Docker website.
  2. Clone the Shuffler Repository: Use Git to clone the Shuffler repository from GitHub: 
    git clone https://github.com/frikky/Shuffle
cd Shuffle
  3. Configure the Environment: Copy the .env.example file to .env and modify the environment variables to suit your needs. Pay close attention to the database credentials and API keys. 
    cp .env.example .env
nano .env

    Key variables to configure:

    • POSTGRES_USER: The PostgreSQL username.
    • POSTGRES_PASSWORD: The PostgreSQL password.
    • SHUFFLER_SECRET_KEY: A randomly generated secret key for signing cookies. Generate a complex random string.
    • SHUFFLER_SUPERUSER_EMAIL: Email for the admin user creation
    • SHUFFLER_SUPERUSER_PASSWORD: Password for the admin user creation
  4. Start Shuffler with Docker Compose: Run the following command to build and start the Shuffler containers: 
    docker-compose up -d
  5. Access Shuffler: Open your web browser and navigate to http://localhost:8000 (or the appropriate IP address and port based on your Docker configuration).

After the first start, create the admin user using docker-compose exec command

docker-compose exec backend python manage.py createsuperuser --username admin --email SHUFFLER_SUPERUSER_EMAIL --password SHUFFLER_SUPERUSER_PASSWORD

Usage: Building a Simple Workflow

A powerful nighttime explosion at a factory with flames and smoke under power lines.
A powerful nighttime explosion at a factory with flames and smoke under power lines.

Let’s create a simple workflow that retrieves the IP address information using an external API.

  1. Access the Shuffler UI: Log in to the Shuffler web interface.
  2. Create a New Workflow: Click the “Workflows” tab and then the “New Workflow” button. Give your workflow a descriptive name, such as “IP Information Lookup.”
  3. Add a Trigger: Drag and drop a “Manual Trigger” app from the left sidebar onto the workflow canvas. This trigger will allow you to manually start the workflow.
  4. Add an Input: Configure the Manual Trigger to accept an IP address as input. Add a field called “ip_address” with type “string”.
  5. Add an HTTP Request App: Search for “HTTP Request” in the app store and drag it onto the canvas. Connect the Manual Trigger to the HTTP Request app.
  6. Configure the HTTP Request App: Configure the HTTP Request app to send a GET request to an IP information API. Many free APIs are available online, such as https://ipinfo.io/{{ip_address}}. Set the ip_address parameter value to the input from your manual trigger.
  7. Add a Parse JSON App: Drag a “Parse JSON” app onto the canvas and connect it to the HTTP Request app. This app will parse the JSON response from the API.
  8. Add a Log App: Drag a “Log” app onto the canvas and connect it to the Parse JSON app. Configure the Log app to print the parsed JSON data. This will display the IP information in the workflow execution logs. You can also use other apps like sending an email or posting a Slack message.
  9. Run the Workflow: Click the “Run” button. Enter an IP address in the input field (e.g., 8.8.8.8) and click “Execute”.
  10. View the Results: Check the workflow execution logs to see the IP information retrieved from the API.

Here’s an example of the HTTP Request configuration:


  {
    "method": "GET",
    "url": "https://ipinfo.io/{{ip_address}}",
    "headers": {
      "Content-Type": "application/json"
    }
  }

Tips & Best Practices

Firefighters on a ladder extinguishing a huge building fire, emitting thick clouds of smoke into the sky.
Firefighters on a ladder extinguishing a huge building fire, emitting thick clouds of smoke into the sky.
  • Use Meaningful Names: Give your workflows, apps, and variables descriptive names to improve readability and maintainability.
  • Leverage Comments: Add comments to your workflows to explain the purpose of each step. This is especially important for complex workflows.
  • Error Handling: Implement error handling mechanisms to gracefully handle failures in your workflows. Use the “Try/Catch” app to catch exceptions and prevent workflows from crashing.
  • Reusable Subworkflows: Create reusable subworkflows for common tasks. This promotes code reuse and simplifies workflow design.
  • Secure Credentials: Use environment variables or a secrets management solution to store sensitive credentials, such as API keys and passwords. Do not hardcode credentials in your workflows.
  • Regularly Update Shuffler: Keep your Shuffler installation up-to-date to benefit from the latest features, bug fixes, and security patches.
  • Explore the App Store: Shuffler has a growing app store with integrations to various security tools and services. Explore the app store to find apps that can enhance your workflows.

Consider the following to improve Security Operations Center (SOC) effectiveness:

  • Prioritize Alerts: Implement workflows that automatically prioritize security alerts based on severity and impact.
  • Automate Threat Intelligence: Integrate Shuffler with threat intelligence feeds to enrich security data and identify potential threats.
  • Streamline Incident Response: Automate incident response tasks, such as isolating infected systems, blocking malicious IP addresses, and notifying stakeholders.
  • Improve Collaboration: Use Shuffler to facilitate collaboration among security analysts and other teams. Share workflows and incident reports to improve communication and knowledge sharing.

Troubleshooting & Common Issues

Powerful drone in flight with cargo against dramatic cloudy background.
Powerful drone in flight with cargo against dramatic cloudy background.
  • Workflow Errors: Check the workflow execution logs for error messages and stack traces. These logs can provide valuable insights into the cause of the error.
  • App Connection Issues: Verify that you have correctly configured the credentials and API keys for each app. Ensure that the app is able to connect to the target service or tool.
  • Database Connectivity Issues: Check the database connection settings in the .env file. Ensure that the database server is running and accessible.
  • Performance Issues: If you experience performance issues, try increasing the resources allocated to the Shuffler containers. You can also optimize your workflows to reduce the number of API calls and data processing steps.
  • Version Conflicts: Ensure that all apps and dependencies are compatible with the Shuffler version you are using. Check the app documentation for compatibility information.
  • “Workflow Won’t Execute”: Ensure the trigger is correctly configured. It will have the “Error” icon if the inputs are not configured properly.

Example: Troubleshooting a failed API call in a workflow.

Problem: The workflow fails with an error message indicating that an API call failed.

Troubleshooting Steps:

  1. Check the Workflow Logs: Examine the workflow execution logs for detailed error messages. Look for HTTP status codes (e.g., 404 Not Found, 500 Internal Server Error) and any error messages returned by the API.
  2. Verify the API Endpoint: Double-check the API endpoint URL in the HTTP Request app. Ensure that it is correct and that the API is accessible from the Shuffler server.
  3. Check the Request Headers: Verify that the request headers (e.g., Content-Type, Authorization) are correctly configured. Some APIs require specific headers.
  4. Inspect the Request Body: If the API call involves sending data in the request body (e.g., POST or PUT request), inspect the request body to ensure that it is correctly formatted and contains the required data.
  5. Test the API Call Manually: Use a tool like curl or Postman to test the API call manually. This can help you isolate the problem and determine whether it is related to the Shuffler workflow or the API itself.

FAQ

Detailed view of a drone part featuring glowing red LED lights and carbon fiber arm.
Detailed view of a drone part featuring glowing red LED lights and carbon fiber arm.
Q: What is Shuffler?
A: Shuffler is an open-source security automation and orchestration (SOAR) platform that enables security teams to automate incident response processes and correlate security data.
Q: What are the benefits of using Shuffler?
A: Shuffler improves efficiency, reduces manual effort, enhances threat detection and response, and fosters collaboration among security teams.
Q: Is Shuffler difficult to install and use?
A: Shuffler offers multiple installation options, with Docker Compose being the simplest. The visual workflow editor makes it relatively easy to design and execute automated tasks.
Q: What kind of integrations does Shuffler have?
A: Shuffler integrates with a wide range of security tools and data sources through its app store, including SIEMs, firewalls, threat intelligence platforms, and ticketing systems.
Q: Is Shuffler really free?
A: Yes, Shuffler is an open-source tool and it is free to use under the Apache 2.0 license.

Conclusion

Shuffler represents a powerful and accessible solution for security automation and orchestration. Its open-source nature, intuitive interface, and extensive integration capabilities make it a valuable asset for security teams of all sizes. Whether you’re looking to automate incident response, correlate security data, or improve collaboration, Shuffler can help you achieve your goals. Give Shuffler a try today and experience the benefits of security automation! Visit the official Shuffler GitHub repository to download and get started: https://github.com/frikky/Shuffle

Leave a Comment