DIY: your own Dependency Injection library!
Demystifying the internals of DI libraries
Dependency Injection libraries are powerful tools, but they're often also intimidating & confusing.
When that happens to me, I find that understanding how a tool works helps me get over the initial scare of the dark magic internals.
In this article, I'll walk you through how to implement your own dependency injection library. Starting with manual dependency injection, we'll progressively build a simplistic version of Google Guice, then Dagger 1 and eventually Dagger 2.
By the end of this article, I'm hoping you'll have built up a good intuition for how all these libraries work under the hood. You'll be the life of the party when you casually drop with a straight face: "oh yeah, a dependency injection library is mostly just a map of types to factories".
The code presented here is available at github.com/pyricau/diy.
Manual Dependency Injection
Dependency Injection is a pattern, so starting with no library is helpful. Manual dependency injection typically requires creating instances in the right order, in a dedicated configuration place in the code.
Coffee Example
We want to create a CoffeeMaker
, which needs a CoffeeLogger
, a Heater
and a Pump
. For the Heater
we'll use an ElectricHeater
which also needs a CoffeeLogger
, and for the Pump
we'll use a Thermosiphon
which needs a CoffeeLogger
and a Heater
.
Here's what that looks like in Kotlin:
class CoffeeLogger
interface Heater
class ElectricHeater(logger: CoffeeLogger) : Heater
interface Pump
class Thermosiphon(logger: CoffeeLogger, heater: Heater) : Pump
class CoffeeMaker(logger: CoffeeLogger, heater: Heater, pump: Pump)
We can represent the CoffeeMaker
and its dependencies as a directed graph:
A dependency graph is actually a Directed Acyclic Graph aka DAG (hence the name Dagger!) as there cannot be cycles between dependencies.
CoffeeMaker
here is called an Entry Point, it's the thing we want to build and the root of our dependency graph.
Let's create a CoffeeMaker
and brew!
val logger = CoffeeLogger()
val heater: Heater = ElectricHeater(logger)
val pump: Pump = Thermosiphon(logger, heater)
val coffeeMaker = CoffeeMaker(logger, heater, pump)
coffeeMaker.brew()
With manual Dependency Injection, creating dependencies in the right order quickly becomes a problem as the number of collaborators increases. To avoid these issues, we need a Dependency Injection library!
Concepts
Let's first introduce a few API contracts that will be useful throughout this article.
ObjectGraph
The ObjectGraph
is our entry point into a DI library. It's also known as Injector
, Container
, or Component
. It's what our application code uses to get started with doing things, and its main job is to provide instances of a requested type:
class ObjectGraph {
operator fun <T> get(requestedType: Class<T>): T
}
The API is straightforward:
val coffeeMaker = objectGraph.get(CoffeeMaker::class.java)
// Or using the get() operator overload:
val coffeeMaker = objectGraph[CoffeeMaker::class.java]
We can write a reified extension function to leverage the power of the Kotlin compiler:
inline fun <reified T> ObjectGraph.get() = get(T::class.java)
// Thank you Kotlin compiler!
val coffeeMaker = objectGraph.get<CoffeeMaker>()
Factory
A Factory
knows how to create instances of a particular type. It can leverage the ObjectGraph
to retrieve the dependencies needed to create a collaborator.
fun interface Factory<T> {
fun get(objectGraph: ObjectGraph): T
}
The Factory
for CoffeeMaker
could be implemented as:
val coffeeMakerFactory = Factory { objectGraph ->
CoffeeMaker(objectGraph.get(), objectGraph.get(), objectGraph.get())
}
CoffeeMaker(logger, heater, pump)
here and can just repeatedly call the reified
function ObjectGraph.get()
, the Kotlin compiler will then pass in the right Class
objects.Module
A Module
knows how to create a factory for a specific type. Module.get()
might return null if a given module doesn't know how to create a factory for that requested type.
interface Module {
operator fun <T> get(requestedType: Class<T>): Factory<T>?
}
At this point, we start seeing how these concepts connect: when calling ObjectGraph.get()
, the object graph will leverage its list of Module
to find a suitable Factory
for that type and then use the Factory
to create the instance.
FactoryHolderModule
Our initial Module
implementation is FactoryHolderModule
, it holds a map of types to their associated Factory
. We call FactoryHolderModule.install(type, factory)
to add new factory, and FactoryHolderModule.get(type)
to retrieve it:
class FactoryHolderModule : Module {
private val factories = mutableMapOf<Class<out Any?>, Factory<out Any?>>()
override fun <T> get(requestedType: Class<T>): Factory<T>? =
factories[requestedType] as Factory<T>?
fun <T> install(
requestedType: Class<T>,
factory: Factory<T>
) {
factories[requestedType] = factory
}
}
Here's how we would add the CoffeeMaker
factory to a FactoryHolderModule
:
val module = FactoryHolderModule()
module.install(CoffeeMaker::class.java) { objectGraph ->
CoffeeMaker(objectGraph.get(), objectGraph.get(), objectGraph.get())
}
Let's make this API nicer! We don't like having to pass in CoffeeMaker::class.java
. Also, repeating objectGraph
is annoying, could we use a lambda with receiver instead?
inline fun <reified T> FactoryHolderModule.install(
noinline factory: ObjectGraph.() -> T
) = install(T::class.java, factory)
// Nicer!
val module = FactoryHolderModule()
module.install {
CoffeeMaker(get(), get(), get())
}
ObjectGraph
implementation
Our ObjectGraph
takes in a list of Module
that knows how to create factories. ObjectGraph.get()
retrieves the factory from the modules and then calls Factory.get(ObjectGraph)
:
class ObjectGraph(private val modules: List<Module>) {
constructor(vararg modules: Module) : this(modules.asList())
operator fun <T> get(requestedType: Class<T>): T {
val factory = modules
.firstNotNullOf { module -> module[requestedType] }
return factory.get(this)
}
}
Delegating to the provided modules on every call to ObjectGraph.get()
could be wasteful, so we can leverage FactoryHolderModule
to add a caching layer in ObjectGraph
for the factories:
class ObjectGraph(private val modules: List<Module>) {
constructor(vararg modules: Module) : this(modules.asList())
// Cache of factories already retrieves from modules.
private val factoryHolder = FactoryHolderModule()
operator fun <T> get(requestedType: Class<T>): T {
val knownFactoryOrNull = factoryHolder[requestedType]
val factory = knownFactoryOrNull ?: modules
.firstNotNullOf { module -> module[requestedType] }
.also { factory ->
factoryHolder.install(requestedType, factory)
}
return factory.get(this)
}
}
Putting it all together
Let's create a CoffeeMaker
and brew! We can install our factories on a FactoryHolderModule
, then create an ObjectGraph
with that module and ask it for a CoffeeMaker
instance.
val module = FactoryHolderModule()
module.install {
CoffeeLogger()
}
module.install {
ElectricHeater(get())
}
module.install {
Thermosiphon(get(), get())
}
module.install {
CoffeeMaker(get(), get(), get())
}
val objectGraph = ObjectGraph(module)
val coffeeMaker = objectGraph.get<CoffeeMaker>()
coffeeMaker.brew()
Unfortunately, this doesn't work! CoffeeMaker
needs a Heater
and a Pump
. We've added a factory for Thermosiphon
which is a Pump
and ElectricHeater
which is a Heater
, but we didn't connect the interfaces with their implementations.
Bind
Let's introduce a bind()
function that associates a requested type to a factory of a provided subtype:
inline fun <reified REQUESTED, reified PROVIDED : REQUESTED>
FactoryHolderModule.bind() {
install(REQUESTED::class.java) { objectGraph ->
objectGraph[PROVIDED::class.java]
}
}
// Nice!
module.bind<Heater, ElectricHeater>()
module.bind<Pump, Thermosiphon>()
Singletons
CoffeeMaker
and Thermosiphon
both need a Heater
. The CoffeeMaker
turns the Heater
on, and the Thermosiphon
starts pumping if the Heater
is hot. For things to work correctly, CoffeeMaker
and Thermosiphon
should use the same Heater
instance. We need singleton support!
Let's create a function that transforms any Factory
into a caching factory that will reuse the instance after the first call:
fun <T> singleton(factory: Factory<T>): Factory<T> {
var instance: Any? = UNINITIALIZED
return Factory { linker ->
if (instance === UNINITIALIZED) {
instance = factory.get(linker)
}
instance as T
}
}
val UNINITIALIZED = Any()
We already have a nice install
function that takes a lambda with receiver, let's create a variant for singletons:
inline fun <reified T> FactoryHolderModule.installSingleton(
noinline factory: ObjectGraph.() -> T
) {
install(T::class.java, singleton(factory))
}
It works!
We've connected interfaces to implementations and added singletons:
val module = FactoryHolderModule()
module.bind<Heater, ElectricHeater>()
module.bind<Pump, Thermosiphon>()
module.installSingleton {
CoffeeLogger()
}
module.installSingleton {
ElectricHeater(get())
}
module.install {
Thermosiphon(get(), get())
}
module.install {
CoffeeMaker(get(), get(), get())
}
val objectGraph = ObjectGraph(module)
val coffeeMaker = objectGraph.get<CoffeeMaker>()
coffeeMaker.brew()
Ugh, that's a lot more boilerplate than our manual DI:
val logger = CoffeeLogger()
val heater: Heater = ElectricHeater(logger)
val pump: Pump = Thermosiphon(logger, heater)
val coffeeMaker = CoffeeMaker(logger, heater, pump)
coffeeMaker.brew()
Can we get rid of the boilerplate?
ReflectiveModule
- Guice style
What if we used reflection to figure out how to create object instances?
@Inject
First, we need a way to indicate which constructor to call, and convey which instances should be singletons. We can leverage the javax.inject
library, which provides the @Inject
and @Singleton
annotations:
dependencies {
// ...
api("javax.inject:javax.inject:1")
}
Let's sprinkle our annotations:
import javax.inject.Inject
import javax.inject.Singleton
@Singleton
class ElectricHeater @Inject constructor(
private val logger: CoffeeLogger
) : Heater {
// ...
}
Injected constructor
For a given class to inject, we use reflection to find the constructor annotated with @Inject
:
val requestedType: Class<T> = //...
val injectConstructor = requestedType.constructors.single {
it.isAnnotationPresent(Inject::class.java)
}
We extract the types of the constructor parameters, ask the ObjectGraph
for an instance of each parameter type then pass these parameters to the constructor:
val objectGraph: ObjectGraph = // ...
val parameters = injectConstructor.parameterTypes.map { paramType ->
objectGraph[paramType]
}.toTypedArray()
val instance = injectConstructor.newInstance(*parameters)
ReflectiveFactory
All together, we get a ReflectiveFactory
:
class ReflectiveFactory<T>(
requestedType: Class<T>
) : Factory<T> {
private val injectConstructor = requestedType.constructors.single {
it.isAnnotationPresent(Inject::class.java)
} as Constructor<T>
override fun get(objectGraph: ObjectGraph): T {
val parameters = injectConstructor.parameterTypes.map { paramType ->
objectGraph[paramType]
}.toTypedArray()
return injectConstructor.newInstance(*parameters)
}
}
Then we create a ReflectiveModule
that creates the right ReflectiveFactory
for each requested type. It also checks if the class is annotated with @Singleton
, in which case it wraps the factory in a caching factory:
class ReflectiveModule : Module {
override fun <T> get(requestedType: Class<T>): Factory<T> {
val reflectiveFactory = ReflectiveFactory(requestedType)
return if (requestedType.isAnnotationPresent(Singleton::class.java)) {
singleton(reflectiveFactory)
} else {
reflectiveFactory
}
}
}
Less boilerplate!
Our coffee example looks a lot nicer, it's similar to how Google Guice works:
val bindingModule = FactoryHolderModule().apply {
bind<Heater, ElectricHeater>()
bind<Pump, Thermosiphon>()
}
val objectGraph = ObjectGraph(
bindingModule,
ReflectiveModule()
)
val coffeeMaker = objectGraph.get<CoffeeMaker>()
coffeeMaker.brew()
This works well, but object creation is done through reflection which is slow. Could we generate code instead?
InjectProcessor
β Dagger-1 style
Generated factories
What if we generated the factory for each injected object, at compile time:
class Thermosiphon_Factory : Factory<Thermosiphon> {
override fun get(objectGraph: ObjectGraph) = Thermosiphon(
objectGraph.get(),
objectGraph.get()
)
}
We'd also need to implement singleton support in the generated factories, leveraging the singleton
function we defined earlier that transforms any Factory
into a caching factory:
class ElectricHeater_Factory : Factory<ElectricHeater> {
private val singletonFactory = singleton { objectGraph ->
ElectricHeater(
objectGraph.get()
)
}
override fun get(objectGraph: ObjectGraph) = singletonFactory
.get(objectGraph)
}
InjectProcessor
To generate the factory classes, we can use KSP. This article is already long so I won't bore you with all the details (read the source), here's how we generate the factory classes:
val className = "${injectedClassSimpleName}_Factory"
ktFile.appendLine("class $className : Factory<$injectedClassSimpleName> {")
val constructorInvocation =
"${injectedClassSimpleName}(" + function.parameters.joinToString(", ") {
"objectGraph.get()"
} + ")"
if (injectedClass.isAnnotationPresent(Singleton::class)) {
ktFile.appendLine(" private val singletonFactory = singleton { objectGraph ->")
ktFile.appendLine(" $constructorInvocation")
ktFile.appendLine(" }")
ktFile.appendLine()
ktFile.appendLine(
" override fun get(objectGraph: ObjectGraph) = singletonFactory.get(objectGraph)"
)
} else {
ktFile.appendLine(
" override fun get(objectGraph: ObjectGraph) = $constructorInvocation"
)
}
ktFile.appendLine("}")
InjectProcessorModule
We still need to use reflection to create an instance for each generated factory class:
class InjectProcessorModule : Module {
override fun <T> get(requestedType: Class<T>) : Factory<T> {
val factoryClass = Class.forName("${requestedType.name}_Factory")
val factoryConstructor = factoryClass.getDeclaredConstructor()
return factoryConstructor.newInstance() as Factory<T>?
}
}
The generated factory will create objects without any reflection involved.
Less reflection!
Our coffee example runs faster, and the setup is almost identical, although we have to enable KSP:
plugins {
id("com.google.devtools.ksp")
kotlin("jvm")
}
dependencies {
// ...
ksp(project(":diy-processor"))
}
The result is similar to how Dagger 1 works:
val objectGraph = ObjectGraph(
FactoryHolderModule().apply {
bind<Heater, ElectricHeater>()
bind<Pump, Thermosiphon>()
},
InjectProcessorModule()
)
val coffeeMaker = objectGraph.get<CoffeeMaker>()
coffeeMaker.brew()
Could we remove the last remaining use of reflection, get rid of the map of factories, and just invoke the right generated code as needed?
ComponentProcessor
β Dagger-2 style
We want to generate code that doesn't use reflection at all. To do this, we need a way to define what instance our object graph should be able to provide. We really only care about one thing: retrieving CoffeeMaker
instances.
The dependency graph is a Directed Acyclic Graph, and CoffeeMaker
is its root, which we call an entry point. We can resolve the entire dependency graph by looking at CoffeeMaker
dependencies and then recursively looking at the dependencies of these dependencies. And we can do all that at compile time!
@Component
interface
Let's define an interface that provides CoffeeMaker
instances:
@Component
interface CoffeeComponent {
val coffeeMaker: CoffeeMaker
}
KSP ComponentProcessor
We then create a KSP ComponentProcessor
to find this interface at compile time:
val symbols = resolver.getSymbolsWithAnnotation(Component::class.java.name)
val componentInterfaces = symbols
.filterIsInstance<KSClassDeclaration>()
.filter { it.validate() && it.classKind == INTERFACE }
We can look for properties on that interface, which we'll call entry points:
fun readEntryPoints(classDeclaration: KSClassDeclaration) =
classDeclaration.getDeclaredProperties().map { property ->
val resolvedPropertyType = property.type.resolve().declaration
EntryPoint(property, resolvedPropertyType)
}.toList()
For each of these entry points class, we can look for an @Inject
constructor, list the constructor parameters, then look for @Inject
constructors for these parameters as well, etc. Here's the code, it might seem like a lot of code but at the core it's a while loop that explores the dependency graph from the entry points:
fun traverseDependencyGraph(factoryEntryPoints: List<KSDeclaration>):
List<ComponentFactory> {
val typesToProcess = mutableListOf<KSDeclaration>()
typesToProcess += factoryEntryPoints
val factories = mutableListOf<ComponentFactory>()
val typesVisited = mutableListOf<KSDeclaration>()
while (typesToProcess.isNotEmpty()) {
val visitedClassDeclaration = typesToProcess.removeFirst()
as KSClassDeclaration
if (visitedClassDeclaration !in typesVisited) {
typesVisited += visitedClassDeclaration
val injectConstructors = visitedClassDeclaration.getConstructors()
.filter { it.isAnnotationPresent(Inject::class) }
.toList()
check(injectConstructors.size < 2) {
"There should be a most one @Inject constructor"
}
if (injectConstructors.isNotEmpty()) {
val injectConstructor = injectConstructors.first()
val constructorParams = injectConstructor.parameters.map {
it.type.resolve().declaration
}
typesToProcess += constructorParams
val isSingleton = visitedClassDeclaration
.isAnnotationPresent(Singleton::class)
factories += ComponentFactory(
visitedClassDeclaration,
constructorParams,
isSingleton
)
}
}
}
return factories
}
@Binds
While the above code takes care of classes annotated with @Inject
and @Singleton
, remember that we also need a way to bind Heater
to ElectricHeater
and Pump
to Thermosiphon
. But we don't have a module to call methods on anymore, all we have is our component interface.
So we'll do the same weird trick that Dagger 2 did: define a new interface that will never be implemented, and only exists to hold methods that will never be invoked. These methods are our compile time API for defining an association between an interface and its implementation:
@Component(modules = [CoffeeBindsModule::class])
interface CoffeeComponent {
val coffeeMaker: CoffeeMaker
}
interface CoffeeBindsModule {
@Binds fun bindHeater(heater: ElectricHeater): Heater
@Binds fun bindPump(pump: Thermosiphon): Pump
}
We can now read the binding modules at compile time in our ComponentProcessor
and build a map of requested types to provided types:
fun readBinds(componentAnnotation: KSAnnotation):
Map<KSDeclaration, KSDeclaration> {
val bindModules = componentAnnotation
.getArgument("modules")
.value as List<KSType>
val binds = bindModules
.map { it.declaration as KSClassDeclaration }
.flatMap { it.getDeclaredFunctions() }
.filter { it.isAnnotationPresent(Binds::class) }
.associate { function ->
val resolvedReturnType = function.returnType!!
.resolve().declaration
val resolvedParamType = function.parameters
.single().type.resolve().declaration
resolvedReturnType to resolvedParamType
}
return binds
}
Generating the component implementation
All that's left for us is to generate the component implementation:
fun generateComponent(
model: ComponentModel,
ktFile: OutputStream
) {
with(model) {
ktFile.appendLine("package $packageName")
ktFile.appendLine()
imports.forEach { import ->
ktFile.appendLine("import $import")
}
ktFile.appendLine()
ktFile.appendLine("class $className : $componentInterfaceName {")
factories.forEach { (classDeclaration, parameterDeclarations, isSingleton) ->
val name = classDeclaration.simpleName.asString()
val parameters = parameterDeclarations.map { requestedType ->
val providedType = binds[requestedType] ?: requestedType
providedType.simpleName.asString()
}
val singleton = if (isSingleton) "componentSingleton " else ""
ktFile.appendLine(" private val provide$name = $singleton{")
ktFile.appendLine(
" $name(${parameters.joinToString(", ") { "provide$it()" }})"
)
ktFile.appendLine(" }")
}
entryPoints.forEach { (propertyDeclaration, type) ->
val name = propertyDeclaration.simpleName.asString()
val typeSimpleName = type.simpleName.asString()
ktFile.appendLine(" override val $name: $typeSimpleName")
ktFile.appendLine(" get() = provide$typeSimpleName()")
}
ktFile.appendLine("}")
}
}
This generates the following CoffeeComponent
implementation:
package coffee
import diy.componentSingleton
class GeneratedCoffeeComponent : CoffeeComponent {
private val provideCoffeeMaker = {
CoffeeMaker(
provideCoffeeLogger(),
provideElectricHeater(),
provideThermosiphon()
)
}
private val provideElectricHeater = componentSingleton {
ElectricHeater(provideCoffeeLogger())
}
private val provideThermosiphon = {
Thermosiphon(provideCoffeeLogger(), provideElectricHeater())
}
private val provideCoffeeLogger = componentSingleton {
CoffeeLogger()
}
override val coffeeMaker: CoffeeMaker
get() = provideCoffeeMaker()
}
Notice how there's no mention of Pump
or Heater
in this code, instead the factories are directly retrieving the appropriate implementation.
No more reflection!
Let's create a CoffeeMaker
and brew!
val component = GeneratedCoffeeComponent()
val coffeeMaker = component.coffeeMaker
coffeeMaker.brew()
A different approach to binding types
After I reading this article, Manu Sridharan reached out with some feedback (thanks!) and an interesting question: "Regarding the "weird trick" for bindings from Dagger 2, it might be interesting to suggest why they did it this way. My guess is because of limitations in what types of arguments you can pass into a Java annotation."
I'm not sure why the Dagger 2 team decided to use abstract methods to define bindings, but I thought it'd be an interesting experiment to try an alternative approach.
Repeatable annotation
First, I defined a new @Bind
annotation (to differentiate from @Binds
, no s
) and decided each annotation would define a single binding from a requested type to a provided type. Since we'll need more than binding, I made it repeatable:
@Repeatable
@Target(CLASS)
@Retention(SOURCE)
annotation class Bind(
val requested: KClass<*>,
val provided: KClass<*>
)
Our CoffeeBindsModule
can now be updated:
@Bind(
requested = Heater::class,
provided = ElectricHeater::class
)
@Bind(
requested = Pump::class,
provided = Thermosiphon::class
)
interface CoffeeBindsModule
While the contract is a lot clearer, that feels more verbose than the previous approach:
interface CoffeeBindsModule {
@Binds fun bindHeater(heater: ElectricHeater): Heater
@Binds fun bindPump(pump: Thermosiphon): Pump
}
Annotation type parameters?
Then I thought: it'd be nice if I could enforce that provided
has to extend requested
. I wonder if annotations can have type parameters?
Turns out, they can!
@Repeatable
@Target(CLASS)
@Retention(SOURCE)
annotation class Bind<REQUESTED : Any, PROVIDED : REQUESTED>(
val requested: KClass<REQUESTED>,
val provided: KClass<PROVIDED>
)
Here our updated module:
@Bind<Heater, ElectricHeater>(
requested = Heater::class,
provided = ElectricHeater::class
)
@Bind<Pump, Thermosiphon>(
requested = Pump::class,
provided = Thermosiphon::class
)
interface CoffeeBindsModule
Only type parameters
But wait a minute: if I'm providing type arguments to the annotation, then I can read those types at compile time, and I don't need the annotation arguments!
@Repeatable
@Target(CLASS)
@Retention(SOURCE)
annotation class Bind<REQUESTED : Any, PROVIDED : REQUESTED>
The result looks really nice:
@Bind<Heater, ElectricHeater>
@Bind<Pump, Thermosiphon>
interface CoffeeBindsModule
One nice benefit is that the IDE can now surface binding type errors as we type:
This new @Bind
annotation would be so much better than @Binds
from Dagger 2:
Less boilerplate
Easier to use: the annotation requires two parameters, with names that make it clear which is which.
Less prone to errors, you can't get the types wrong or pass in too many parameters.
No more weird abstract methods that are never called or implemented.
Unfortunately, Dagger 2 is a java annotation processor and, unlike Kotlin, Java annotations cannot have type parameters:
@Bind
Implementation
The KSP implementation is straightforward, we update our annotation processor to read the annotation type arguments instead of the interface declared methods:
fun readBinds(componentAnnotation: KSAnnotation):
Map<KSDeclaration, KSDeclaration> {
val bindModules = componentAnnotation
.getArgument("modules")
.value as List<KSType>
val binds = bindModules
.map { it.declaration as KSClassDeclaration }
.flatMap { it.annotations }
.filter { it isInstance Bind::class }
.associate { annotation ->
val annotationArguments = annotation
.annotationType.resolve().arguments
val requested = annotationArguments.first()
.type!!.resolve().declaration
val provided = annotationArguments.last()
.type!!.resolve().declaration
requested to provided
}
return binds
}
Conclusion
The code presented in this article is available at github.com/pyricau/diy, feel free to experiment with it! Who knows, you might end up creating Dagger 3 (if you do, hit me up, I have feature requests π).