Extracting DB layer when introducing more applications to the project

[Algebra8]

Hello @benbjohnson

First and foremost, this is an amazing resource and thank you for publishing it 🙏

My question is going to be related to #13, specifically the second situation that you mentioned in your answer:

Your extracted library is being shared by two applications that share the same application domain. I find this happens in microservice applications a lot. An example would be an accounting microservice and an e-commerce service both depend on a Customer type and related functionality.

I find the fact that the DB layer does the heavy lifting for implementing a service contract interesting and was wondering if/how you would change it under the circumstances described above?

For example, if we decided to expand the application wtf to include some new features (e.g. lol) and it warranted being extracted into its own application:

cmd/
	wtf/							# Contains all binaries related to wtf
		wtf-storybook/
		wtfa/  						# Previously known as wtf
		wtfd/
	lol/							# Contains all binaries related to lol
		lol-storybook/
		lold/
internal/
	wtf/							# Contains everything that was in the root
		sqlite/
		...
	lol/
		sqlite/
		...

And, assuming that there is shared functionality between the sqlite implementations (such as FindOne, FindMany, etc), would you consider changing the naming?

For example, would you have something like:

cmd/
	wtf/
		wtf-storybook/
		wtfa/
		wtfd/
	lol/
		lol-storybook/
		lold/
internal/
	wtf/
		service/
		...
	lol/
		service/
                ...
	sqlite/

Where package service might contain a struct that holds a DB dependency:

// internal/wtf/service/auth.go

// package service was formally known as package sqlite. Ever since the lol
// app was added, the sqlite implementation was pushed to its own library and
// service makes use of it.
//
// service layer objects will rely on an injected DB object (could be sqlite)
// to perform CRUD operations.
package service

import (
	".../internal/db"
	".../internal/wtf"
)

// DB defines the contract that any underlying database implementation will have
// to implement, be it sqlite, mongo, postgres, etc — in order to be used by the
// AuthService.
type DB interface {
	FindAuthByID(context.Context, int) (*wtf.Auth, error)
	AttachAuthAssociations(context.Context, *wtf.Auth) error
}

// AuthService represents a service for managing OAuth authentication.
type AuthService struct {
	db DB
}

// NewAuthService returns a new instance of AuthService attached to DB.
func NewAuthService(db DB) *AuthService {
	return &AuthService{db: db}
}

// FindAuthByID retrieves an authentication object by ID along with the associated user.
// Returns ENOTFOUND if ID does not exist.
func (s *AuthService) FindAuthByID(ctx context.Context, id int) (*wtf.Auth, error) {
	auth, err := db.FindAuthByID(ctx, id)
	if err != nil {
		return nil, err
	}

	if err := db.AttachAuthAssociations(ctx, auth); err != nil {
		return nil, err
	}

	return auth, nil
}

Then in cmd/wtf/wtfd/main.go line 180 (previously was

wtf/cmd/wtfd/main.go

Line 180 in 05bc90c

// Instantiate SQLite-backed services.

), we would have:

	// Instantiate SQLite-backed services.
        sqliteDB := sqlite.New()
	authService := service.NewAuthService(sqliteDB) // sqliteDB implements AuthService.DB
	dialService := service.NewDialService(sqliteDB)  // sqliteDB implements dialService.DB
	dialMembershipService := service.NewDialMembershipService(sqliteDB) // etc
	userService := service.NewUserService(sqliteDB) // etc

So, is this something that you would do in the given situation? Or would you prefer to maintain fine-grain control over each service's use of the underlying DB driver?

Thank you and apologies for the long question 🙏

[benbjohnson]

@Algebra8 No problem on the long question. Thanks for the code examples—they help to give context.

I wouldn't personally abstract out the database itself. I've tried that on projects in the past but there are typically two issues. First, most applications only need use a single type of database (e.g. Postgres) so abstracting it doesn't provide any benefit. Second, abstraction requires that you cater to the lowest common denominator and each database can be quite different in what it provides. For example, many databases provide transactions but some do not. Also, if you do abstract out a DB type then do you also need to abstract out a Tx type?

Another benefit to keeping the abstraction at the service level is that you can implement the service around the database but also implement it at higher levels (e.g. HTTP or a cache layer). That allows you to do something like wrap a memcache.AuthService around a postgres.AuthService to provide caching.

All that being said, if you only have a single implementation and no mocks then you could just avoid interfaces entirely and simply reference the concrete type (e.g. *sqlite.AuthService).

[Algebra8]

Thank you very much for the swift and thoughtful reply :)

These are excellent points and I will keep them in mind.

[luqmansen]

Hi @benbjohnson, I have a question regarding abstracting the database. What's your opinion about abstracting database access to isolate the DB layer from the business logic. If I have very complex logic, then I can just mock the DB on the unit test without actually needing to hit the actual DB, which is faster and more favorable.

[benbjohnson]

@luqmansen I think that abstracting the entire DB is overkill and usually not worth it. I do think that extracting out complex business logic into the top-level domain package can make a lot of sense. Sometimes you need to embed your business logic at lower layers (e.g. SQL) for performance reasons or to deal with implementation details (e.g. transactional semantics offered by your specific database). But if you don't have those concerns then, yeah, push the business logic into the domain.