Early Design Approaches
I first started programming using python to make simple scripts for Houdini and Maya, but quickly moved on to developing more complex toolkits.
I would script an asset type, then create an export button for the toolbar which contained my unified export script. This is called a singleton design pattern, where one class or function encapsulates all functionality. This worked well enough when I was learning, and the types of assets I wanted to create was limited.
However, as I added more asset types to my toolkit, my export script compounded in complexity. The singleton approach became fraught with issues as small changes to any part of the exporter could ripple outwards and disrupt the entire pipeline. Soon my confidence was drained from me. I dared expand or update my code. The singleton exporter had become a monster which consumed my confidence and willingness to experiment with new techniques.
Oh, I forgot to mention. A new exporter would need to be written for every client software I incorporated into the pipeline. That means, Maya, Houdini, Unreal, and Substance would all have their own unwieldy script.
I needed a new approach. I had to think about my code a bit more abstractly.
Behold the Factory
Up to this point in my coding career, I had only ever written scripts, that is to say some class or function designed to accomplish a particular task. The scalability problem I was facing was not a scripting problem per se, nor an issue of grasping the API of a given client application, but a software design problem.
After doing some research on software design patterns, I ended up settling on an abstract factory pattern.
Think of an abstract class like a template. All of the necessary methods and properties are defined in the base class. Any classes which inherit from the base class are required to implement all the abstract methods and properties defined in the parent. For example, exporting is a task which will be done in both Maya and Houdini, but both have separate APIs. Thus the export behavior can be abstracted, and implemented in each application using an abstract class.
Lets look at an example of an abstract class.
from abc import ABC, abstractmethod
class Exporter(ABC):
"""
Abstract class defining our exporter plugin.
This will serve as the template for exporting content from any 3d DCC.
PyPlug will index all of our plugins and look for implementations of
this exporter.
"""
@classmethod
@abstractmethod
def viable(self, **kwargs):
"""
Test for anything in the scene/content which the plugin can
extract.
:return: True if anything viable is found
"""
pass
@classmethod
@abstractmethod
def export(self, directory, **kwargs):
"""
This will export data to external files.
:param directory: Location on disk to save data
:param kwargs: optional keyword data
:return: List of exported files
"""
passNotice the exporter class contains no actual logic, instead broadly defining the pattern of export behavior. Each plug-in that has export functionality simply needs to implement a subclass of this Exporter abstract. The code base is immediately more legible as the python modules are separated along the concerns of both client application and plug-in.
Now comes the factory part of the abstract factory pattern.
import sys, os
from pathlib import Path
import inspect
import importlib
from importlib import util
from ast import ClassDef, parse, ImportFrom
class Factory():
def __init__(self, name="", version="", env_vars="", base_class=None, paths=[]):
#store params
self.paths = paths
for path in self.paths:
sys.path.append(path)
self.base_class = base_class
self.version = version
self.env_vars = env_vars
self.name = name
"""
loop through all of the paths provided and return modules which contain
implementations of the base class
"""
self.plugins = []
self.refresh(paths=self.paths, base_class=self.base_class)
def get(self, plugin_name, refresh=False):
"""
Return a specific plugin based on a name.
"""
return self.plugins[self.plugins.index(plugin_name)]
def available(self):
"""
Return list of available plugins.
"""
return self.plugins
def get_paths(self):
"""
Return directories this factory will index.
"""
return self.paths
def refresh(self, paths, base_class):
"""
This function loops through paths.
For each, parse the file and get a list of defined classes.
Compare the base class parameter to see if the abstract
is defined in the module.
If it is, add it to the list of available plugins.
"""
for p in paths:
for root, dirs, files in os.walk(p, topdown=False):
#ignore pycache folder
if os.path.basename(os.path.normpath(root)) != "__pycache__":
for name in files:
if name.endswith(".py") and name != "__init__.py":
full_file = os.path.join(root, name)
#remove .py from file name
file_name = name[:-3]
#generate module import name
module_name = os.path.basename(os.path.normpath(root))
module_name = module_name+"."+file_name
try:
#parse module
mod = importlib.import_module(module_name)
src = inspect.getsource(mod)
p=parse(inspect.getsource(mod))
#get list of class names defined in the module
names = []
for node in p.body:
if isinstance(node, ClassDef):
names.append(node.name)
elif isinstance(node, ImportFrom):
names.extend(imp.name for imp in node.names)
if base_class in names:
self.register(mod)
except Exception as e:
print("Could not parse module: ", module_name, e)
def register(self, plug_in):
"""
Add a plug in to the list of available plugins.
Quick check to ensure no duplication.
"""
if plug_in not in self.plugins:
self.plugins.append(plug_in)A factory has the simple task of inspecting a python module and storing it if the factory finds an implementation of the desired abstract class. Then, it is a simple matter of looping through the array of modules, and invoking the abstract behavior. This may sound needlessly complicated, but it drastically simplifies plug-in implementation across multiple platforms, as the same factory class can be used in every client application and code is tree-shakable since every plug-in is stored in its own module.
Implementing the factory is consistent across all client applications.
from pyplug import Factory
export_factory = Factory(base_class="Exporter", paths=["path/to/python/modules"])
for p in export_factory.available():
try:
if p.Exporter.viable():
p.Exporter.export("path/to/target/directory")
except Exception as e:
print(str(e))Concluding Thoughts
An abstract factory method is a lightweight and efficient way of simplifying basic client-side plug-in functionality. It allowed me to replace n import statements with n^x lines of interdependent and unstable code with eight lines of perfectly stable code.
This approach is not without its drawbacks however. Since the factory is client-application agnostic, it will parse every plug-in in the target directory regardless of whether it has implemented the target abstract. Plus, each plug-in registered by the factory will perform the viable method whether the contents of the exported asset use the plug-in or not. It is therefore important to keep the viable method as lightweight as possible.
I hope this article saves you from avoidable frustration. Good luck coding going forward.
