5.9 Plugins Updates (BC-SECURITY#760)

* Backwards compatible plugin updates

* file renames

* changelog

* rename example plugin back

* fix docs

* remove csharpserver warning

CHANGELOG.md

@@ -37,14 +37,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 -   Returning tuples from plugin execution functions is deprecated
     -   To return a 400, raise a `PluginValidationException`
     -   To return a 500, raise a `PluginExecutionException`
+-   Loading plugins from a `.plugin` file is deprecated
+    -   Use a `.py` file with a `plugin.yaml` instead
+-   Extending the `Plugin` class is deprecated
+    -   Use the `BasePlugin` class instead
 
 ### Changed
 
 -   Migrated some Pydantic and FastAPI usage away from deprecated features (@Vinnybod)
 -   Updated the install script and Docker file from Python 3.12.0 to 3.12.1 (@Vinnybod)
 -   Upgraded all dependencies with `poetry up` (@Vinnybod)
+-   Plugin updates (@Vinnybod)
+    -   Plugins have a `plugin.yaml`
+    -   Base plugin class is now `BasePlugin`
+    -   Updated plugin documentation
 -   Upgraded Black to 23.12.0 (@Vinnybod)
--   Upgraded Ruff to 0.19.0 (@Vinnybod)
+-   Upgraded Ruff to 0.1.9 (@Vinnybod)
 -   Upgraded Seatbelt to 1.2.1 (@Cx01N)
 
 ## [5.8.4] - 2023-12-22

docs/SUMMARY.md

@@ -31,6 +31,7 @@
   * [Starkiller](interfaces/starkiller/README.md)
     * [Introduction](interfaces/starkiller/introduction.md)
 * [Plugins](plugins/README.md)
+  * [Getting Started](plugins/getting-started.md)
   * [Plugin Development](plugins/plugin-development.md)
   * [Hooks and Filters](plugins/hooks-and-filters.md)
 * [Bypasses](bypasses.md)

docs/plugins/README.md

@@ -1,6 +1,9 @@
 # Plugins
 
-Plugins are an extension of Empire that allow for custom scripts to be loaded. This allows anyone to easily build or add community projects to extend Empire functionality. Plugins can be accessed from the Empire client or the API as long as the plugin follows the [template example](https://github.com/BC-SECURITY/Empire/blob/master/empire/server/plugins/example.py). A list of Empire Plugins is located here.
+Plugins are an extension of Empire that allow for custom scripts to be loaded.
+This allows anyone to build or add community projects to extend Empire functionality.
+Plugins can be accessed from the Empire client or the API as long as the plugin follows
+the [template example](https://github.com/BC-SECURITY/Empire/blob/master/empire/server/plugins/example.py). A list of Empire Plugins is located here.
 
 ## Empire Plugins
 
@@ -15,4 +18,3 @@ Plugins are an extension of Empire that allow for custom scripts to be loaded. T
 | [Nmap-Plugin](https://github.com/BC-SECURITY/Nmap-Plugin)                    | Nmap-Plugin gives a way to interface directly from Empire to [Nmap](https://nmap.org/) and send commands through [Python3-Nmap](https://github.com/nmmapper/python3-nmap).                 |      [![nmap](https://user-images.githubusercontent.com/20302208/120945236-1feb5f80-c6ed-11eb-9ca3-160c66d4c447.gif)](https://user-images.githubusercontent.com/20302208/120945236-1feb5f80-c6ed-11eb-9ca3-160c66d4c447.gif)      | @Cx01N          | X              |
 | [Twilio-Plugin](https://github.com/BC-SECURITY/Twilio-Plugin)                | The Twilio Plugin is meant to show the possibilities of the Hooks feature implemented in Empire 4.1. It sends a text message every time an agent connects.                                 |                                                                                                                                                                                                                                   | @Vinnybod       |                |
 | [denylist-plugin](https://github.com/BC-SECURITY/denylist-plugin)            | The purpose of this plugin is to block certain IP addresses from connecting to the server. It is to showcase the event-driven nature of the hook system.                                   |                                                                                                                                                                                                                                   | @Vinnybod       |                |
-

docs/plugins/getting-started.md

@@ -0,0 +1,30 @@
+# Plugins Getting Started
+
+This page will walk you through the process of creating a plugin for Empire using
+the hello world plugin as an example. The hello world plugin is an example plugin
+that can be found in the `empire/server/plugins/example` directory.
+
+```
+empire/server/plugins/example
+├── __init__.py
+├── example.py
+└── plugin.yaml
+```
+
+The `plugin.yaml` configuration will likely be expanded on in the future, but for now
+it only contains one property: `main`. This is the name of the python file within the
+plugin's directory that contains the plugin class.
+
+```yaml
+main: example.py
+```
+
+The `example.py` file contains the plugin class. The class must be named `Plugin`
+and must inherit from `empire.server.common.plugins.BasePlugin`.
+
+```python
+class Plugin(BasePlugin):
+    ...
+```
+
+To get into the details of the plugin, move onto the [plugin development](./plugin-development.md) page.

docs/plugins/plugin-development.md

@@ -147,6 +147,19 @@ def do_something():
 ## Event-based functionality (hooks and filters)
 This is outlined in [Hooks and Filters](./hooks-and-filters.md).
 
+## Importing other python files
+
+If you want to import other python files in your plugin, you can do so by importing
+them relative to `empire.server.plugins`. For example, if you have a file called
+`example_helpers.py` in the same directory as your plugin, you can import it like so:
+
+```python
+from empire.server.plugins.example import example_helpers
+```
+
+**Note**: Relative imports will not work. For example, the example plugin cannot
+import `example_helpers.py` with `from . import example_helpers`.
+
 ## 4->5 Changes
 Not a lot has changed for plugins in Empire 5.0. We've just added a few guard rails for better
 stability between Empire versions.
@@ -166,9 +179,8 @@ This is no different than the way things were pre 5.0.
 * `plugin_socketio_message` was moved from `MainMenu` to `plugin_service`.
 * Example conversion for a 5.0 plugin can be seen in [ChiselServer-Plugin](https://github.com/BC-SECURITY/ChiselServer-Plugin/compare/5.0)
 
-
 ## Future Work
 * improved plugin logging -
-  Give plugins indidual log files like listeners have. Make those logs accessible via Starkiller.
+  Give plugins individual log files like listeners have. Make those logs accessible via Starkiller.
 * endpoint for installing plugins -
   A user would be able to provide the URL to a git repository and Empire would download and install the plugin.

empire/server/common/plugins.py

@@ -4,7 +4,7 @@
 log = logging.getLogger(__name__)
 
 
-class Plugin:
+class BasePlugin:
     # to be overwritten by child
     def __init__(self, mainMenu):
         # having these multiple messages should be helpful for debugging
@@ -35,3 +35,6 @@ def register(self, mainMenu):
         """Any modifications made to the main menu are done here
         (meant to be overriden by child)"""
         pass
+
+
+Plugin = BasePlugin

empire/server/core/plugin_service.py

@@ -5,7 +5,9 @@
 import os
 import warnings
 from datetime import datetime
+from pathlib import Path
 
+import yaml
 from sqlalchemy import and_, func, or_
 from sqlalchemy.orm import Session, joinedload, undefer
 
@@ -44,24 +46,23 @@ def autostart_plugins(self):
         """
         Autorun plugin commands at server startup.
         """
-        plugins = empire_config.yaml.get("plugins")
-        if plugins:
-            for plugin in plugins:
-                use_plugin = self.loaded_plugins.get(plugin)
-                if not use_plugin:
-                    log.error(f"Plugin {plugin} not found.")
-                    continue
+        plugins = empire_config.yaml.get("plugins", {})
 
-                options = plugins[plugin]
-                req = PluginExecutePostRequest(options=options)
+        for plugin_name, options in plugins.items():
+            use_plugin = self.loaded_plugins.get(plugin_name)
+            if not use_plugin:
+                log.error(f"Plugin {plugin_name} not found.")
+                continue
 
-                with SessionLocal.begin() as db:
-                    results, err = self.execute_plugin(db, use_plugin, req, None)
+            req = PluginExecutePostRequest(options=options)
 
-                if results is False:
-                    log.error(f"Plugin failed to run: {plugin}")
-                else:
-                    log.info(f"Plugin {plugin} ran successfully!")
+            with SessionLocal.begin() as db:
+                results, err = self.execute_plugin(db, use_plugin, req, None)
+
+            if results is False:
+                log.error(f"Plugin failed to run: {plugin_name}")
+            else:
+                log.info(f"Plugin {plugin_name} ran successfully!")
 
     def startup_plugins(self, db: Session):
         """
@@ -70,7 +71,47 @@ def startup_plugins(self, db: Session):
         plugin_path = f"{self.main_menu.installPath}/plugins/"
         log.info(f"Searching for plugins at {plugin_path}")
 
-        # Import old v1 plugins (remove in 5.0)
+        self._legacy_load(plugin_path)
+
+        for directory in os.listdir(plugin_path):
+            plugin_dir = Path(plugin_path) / directory
+
+            if (
+                directory == "example"
+                or not plugin_dir.is_dir()
+                or plugin_dir.name.startswith(".")
+                or plugin_dir.name.startswith("_")
+            ):
+                continue
+
+            plugin_yaml = plugin_dir / "plugin.yaml"
+
+            if not plugin_yaml.exists():
+                log.warning(f"Plugin {plugin_dir.name} does not have a plugin.yaml")
+                continue
+
+            plugin_config = yaml.safe_load(plugin_yaml.read_text())
+            plugin_main = plugin_config.get("main")
+            plugin_file = plugin_dir / plugin_main
+
+            if not plugin_file.is_file():
+                log.warning(f"Plugin {plugin_dir.name} does not have a valid main file")
+                continue
+
+            try:
+                self.load_plugin(plugin_file.name.removesuffix(".py"), plugin_file)
+            except Exception as e:
+                log.error(
+                    f"Failed to load plugin {plugin_file.name}: {e}", exc_info=True
+                )
+
+    def _legacy_load(self, plugin_path):
+        warnings.warn(
+            "Legacy plugin loading will be removed in 6.0",
+            DeprecationWarning,
+            stacklevel=2,
+        )
+
         plugin_names = os.listdir(plugin_path)
         for plugin_name in plugin_names:
             if not plugin_name.lower().startswith(
@@ -93,12 +134,16 @@ def startup_plugins(self, db: Session):
                 ):
                     continue
 
+                warnings.warn(
+                    f"{plugin_name}: Loading plugins from .plugin files is deprecated",
+                    DeprecationWarning,
+                    stacklevel=2,
+                )
                 self.load_plugin(plugin_name, file_path)
 
     def load_plugin(self, plugin_name, file_path):
         """Given the name of a plugin and a menu object, load it into the menu"""
-        # note the 'plugins' package so the loader can find our plugin
-        loader = importlib.machinery.SourceFileLoader(plugin_name, file_path)
+        loader = importlib.machinery.SourceFileLoader(plugin_name, str(file_path))
         module = loader.load_module()
         plugin_obj = module.Plugin(self.main_menu)
 
@@ -108,6 +153,7 @@ def load_plugin(self, plugin_name, file_path):
             if value.get("Strict") is None:
                 value["Strict"] = False
 
+        plugin_name = plugin_obj.info["Name"]
         self.loaded_plugins[plugin_name] = plugin_obj
 
     def execute_plugin(
@@ -134,9 +180,7 @@ def execute_plugin(
             log.warning(
                 f"Plugin {plugin.info.get('Name')} does not support db session or user_id, falling back to old method"
             )
-        except PluginValidationException as e:
-            raise e
-        except PluginExecutionException as e:
+        except (PluginValidationException, PluginExecutionException) as e:
             raise e
         except Exception as e:
             log.error(f"Plugin {plugin.info['Name']} failed to run: {e}", exc_info=True)
@@ -153,9 +197,7 @@ def execute_plugin(
                 )
                 return res
             return res, None
-        except PluginValidationException as e:
-            raise e
-        except PluginExecutionException as e:
+        except (PluginValidationException, PluginExecutionException) as e:
             raise e
         except Exception as e:
             log.error(f"Plugin {plugin.info['Name']} failed to run: {e}", exc_info=True)

empire/server/plugins/basic_reporting/plugin.yaml

@@ -0,0 +1 @@
+main: basic_reporting.py

empire/server/plugins/csharpserver.plugin → empire/server/plugins/csharpserver/csharpserver.py

@@ -1,4 +1,5 @@
 import base64
+import contextlib
 import logging
 import os
 import socket
@@ -44,7 +45,7 @@ def onLoad(self):
         self.tcp_port = 2012
         self.status = "OFF"
 
-    def execute(self, command):
+    def execute(self, command, **kwargs):
         try:
             results = self.do_csharpserver(command)
             return results
@@ -133,13 +134,10 @@ def do_send_message(self, compiler_yaml, task_name, confuse=False):
         b64_task_name = base64.b64encode(bytes_task_name)
 
         # check for confuse bool and convert to string
-        if confuse:
-            bytes_confuse = "true".encode("UTF-8")
-        else:
-            bytes_confuse = "false".encode("UTF-8")
+        bytes_confuse = b"true" if confuse else b"false"
         b64_confuse = base64.b64encode(bytes_confuse)
 
-        deliminator = ",".encode("UTF-8")
+        deliminator = b","
         message = b64_task_name + deliminator + b64_confuse + deliminator + b64_yaml
         s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         s.connect((self.tcp_ip, self.tcp_port))
@@ -166,13 +164,10 @@ def do_send_stager(self, stager, task_name, confuse=False):
         # compiler only checks for true and ignores otherwise
 
         # check for confuse bool and convert to string
-        if confuse:
-            bytes_confuse = "true".encode("UTF-8")
-        else:
-            bytes_confuse = "false".encode("UTF-8")
+        bytes_confuse = b"true" if confuse else b"false"
         b64_confuse = base64.b64encode(bytes_confuse)
 
-        deliminator = ",".encode("UTF-8")
+        deliminator = b","
         message = b64_task_name + deliminator + b64_confuse + deliminator + b64_yaml
         s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         s.connect((self.tcp_ip, self.tcp_port))
@@ -192,11 +187,11 @@ def do_send_stager(self, stager, task_name, confuse=False):
         return file_name
 
     def shutdown(self):
-        try:
-            b64_yaml = base64.b64encode(("dummy data").encode("UTF-8"))
-            b64_confuse = base64.b64encode(("false").encode("UTF-8"))
-            b64_task_name = base64.b64encode(("close").encode("UTF-8"))
-            deliminator = ",".encode("UTF-8")
+        with contextlib.suppress(Exception):
+            b64_yaml = base64.b64encode(b"dummy data")
+            b64_confuse = base64.b64encode(b"false")
+            b64_task_name = base64.b64encode(b"close")
+            deliminator = b","
             message = b64_task_name + deliminator + b64_confuse + deliminator + b64_yaml
             s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
             s.connect((self.tcp_ip, self.tcp_port))
@@ -205,6 +200,5 @@ def shutdown(self):
             self.csharpserverbuild_proc.kill()
             self.csharpserver_proc.kill()
             self.thread.kill()
-        except:
-            pass
+
         return

empire/server/plugins/csharpserver/plugin.yaml

@@ -0,0 +1 @@
+main: csharpserver.py

empire/server/plugins/example.plugin → empire/server/plugins/example/example.py

@@ -3,6 +3,13 @@
 
 from empire.server.common.plugins import Plugin
 
+# Relative imports don't work in plugins right now.
+# from . import example_helpers
+# example_helpers.this_is_an_example_function()
+from empire.server.plugins.example import example_helpers
+
+example_helpers.this_is_an_example_function()
+
 log = logging.getLogger(__name__)
 
 # anything you simply write out (like a script) will run immediately when the
@@ -24,7 +31,7 @@ def onLoad(self):
         self.calledTimes = 0
 
         self.info = {
-            # Plugin Name, at the moment this much match the do_ command
+            # Plugin Name
             "Name": "example",
             # List of one or more authors for the plugin
             "Authors": [
@@ -68,7 +75,7 @@ def execute(self, command):
         try:
             results = self.do_test(command)
             return results
-        except:
+        except Exception:
             return False
 
     def register(self, mainMenu):
@@ -90,7 +97,7 @@ def do_test(self, command):
 
         if self.status == "start":
             self.calledTimes += 1
-            log.info("This function has been called {} times.".format(self.calledTimes))
+            log.info(f"This function has been called {self.calledTimes} times.")
             log.info("Message: " + command["Message"])
 
         else:

empire/server/plugins/example/example_helpers.py

@@ -0,0 +1,2 @@
+def this_is_an_example_function():
+    return True

empire/server/plugins/example/plugin.yaml

@@ -0,0 +1 @@
+main: example.py

empire/server/plugins/reverseshell_stager_server/plugin.yaml

@@ -0,0 +1 @@
+main: reverseshell_stager_server.py