Modules, Packages and Imports

Repositories, Modules, and Packages

• Repository: place in VCS where source code for a project is stored.
• Module: bundle of Go source code that’s distributed and versioned as a single unit. Modules are stored in Repo and contain one-or-more packages.
• Packages: directory of source code which give a module organisation and structure.
• Every module have unique identifier called as module-path. It is usually based on repository where module is stored.
• Example: https://github.com/jonbodner/proteus has module-path of : github.com/jonbodner/proteus

Using go.mod

• A directory of Go source code becomes a module when there is a valid go.mod file in it.
• It should be created using go mod command to manage the module.
• go mod init MODULE_PATH creates go.mod file that makes current directory root of the module. MODULE_PATH should be globally unique name that identifies your module.

// structure of go.mod file

module github.com/mightyjoe781/exmaple // module declaration

go 1.21 // go version determined by toolchain

require (   // dependencies in your build
    github.com/learning-go-book-2e/formatter v0.0.0-20220918024742-18...
    github.com/shopspring/decimal v1.3.1
)

// note: indirect word doesn't have any special meaning here, it just signifies overrides or third-party code
require (
    github.com/fatih/color v1.13.0 // indirect
    github.com/mattn/go-colorable v0.1.9 // indirect 
)

• reading : Go Toolchains

Building Packages

Importing and Exporting

• Go’s import statement allows you to access the exported constants, variables, functions and types in another package.
• Go uses capitalization to determine whether a package-level identifier must be visible outside the package where it is declared. Identifier which starts with lowercase letter or underscore is accessible within the package where it is declared.
• Anything that is exported becomes part of packages API.

Creating and Accessing a Package

• Code

Inside math directory: math.go

package math

func Double(a int) int {
  return a * 2
}

Inside do-format directory: formatter.go

package format

import "fmt"

func Number(num int) string {
  return fmt.Sprintf("The number is %d", num)
}

Now main.go

package main

import (
    "fmt"

    "github.com/learning-go-book-2e/package_example/do-format"
    "github.com/learning-go-book-2e/package_example/math"
)

func main() {
    num := math.Double(2)
    output := format.Number(num) // notice package name determines package name, not dir
    fmt.Println(output)
}

• Sometimes its common for directory names to not match the package name for the sake of versioning.

Naming Packages

• Package names should be descriptive. Rather than util, create a package that describes the functionality. Ex - names.Extract or names.Format
• Avoid repeating the name of package in the names of functions and types. Ex- ExtractName or FormatName in names package is not suitable.

Overriding a Package’s Name

• Sometimes its possible you might end up using two libraries with conflicting package names. Ex- crypto/rand and math/rand
• while importing you can redefine package names as follows

import (
    crand "crypto/rand"
  "fmt"
  "math/rand"
)

Documenting Your Code with Go Doc Comments

• Go Doc format comments are automatically converted into documentation
• There are rules for this
  • place comments directly above the item being documented.
  • start each line with double slashes, followed by a space
  • the first word in the comment for a symbol should be name of the symbol (function, type, constant, variables)
  • use a blank comment line to break your comment into multiple paragraphs.
• To put headers use # and a space after it.
• To link to exported symbol : [pkgName.SymbolName]
• To include text that links to a web page use : [TEXT]: URL format

To display these docs use go doc <pkg_name>. To view the documentation’s HTML formatting before publishing on web use pkgsite (use for pkg.go.dev site)

Using the internal Package

• Sometimes we want to share a function, type or constant among packages in the module but don’t want to make it part of API. We can do this via internal package name.
• When a package is created with name internal the exported identifiers in that package will be available to direct parent packages and siblings of internal

Avoiding Cicular Dependencies

Two of the goals of Go are a fast compiler and easy-to-understand source code. To support this, Go does not allow you to have a circular dependencybetween packages. If package A imports package B, directly or indirectly, package B cannot import package A, directly or indirectly.

Organizing Your Module

recommended guidelines for organizing the modules

• If module is small keep all the code in the single package.
• If module grows and to make some order and making your code more readable, start diving it in two broad categories, main application code and the code which is generic and can be used by the different parts of application.
• If you want your module to be used as a library, the root of your module should have same name as repository.
• Its common for library modules to have one or more appication included with them as utilities, in that case create a directory name cmd
• when developing a library take advantage of internal package. If you create multiple packages within a module and they are outside an internal package, exporting a symbol so it can be used by another package in your module means, it can be used by anyone who uses your module.
• Problem with above use case is with sufficient users of your API, it doesn’t not matter what you promise in contract, all observable behaviour of your system will be depended on by somebody. once that is part of API it should be supported as long there are new version which are not backwards compatible. Read up Hyrum’s Law Link

Simple Go project Layout with Modules

How do you structure your go Apps

Gracefully Renaming and Reorganizing Your API

• To avoid backward-breaking change, don’t remove original identifiers; provide an alternate name instead.
• This is easy with a function or method. If you want to rename or move an exported type, you use an alias. Quite simply, an alias is a new name for a type. type Bar = Foo

Avoiding the init Function if Possible

• Why Go avoids Complex Calls
  • Go does not support method overriding or function overloading to keep code clear and predictable.
• What is init
  • A special function that runs automatically when a package is imported.
  • It does not take arguments or return values.
  • Used to set up package-level state.
• Problems with init
  • Some packages use init for automatic setup, like registering a database driver.
  • Blank imports (_ "package") trigger init without exposing any package functions.
  • This method is now discouraged as it hides important details.

import (
    "database/sql"

    _ "github.com/lib/pq"
)

• Best Practices
  • Use init only for initializing package-level variables that don’t change.
  • If a variable needs to change during runtime, use a struct instead.
• Documentation is Important
  • Since init runs automatically, document its behavior, especially if it performs tasks like file loading or network access.

Working with Modules

Importing Third-Party Code

• Go always builds application from source code into a single binary file. This include your source code and source code that a module depends on.
• We should never use floating-point numbers when we need an exact representation of decimal number. Good option for this is decimal library from shopspring

package main

import (
    "fmt"
  "log"
  "github.com/learning-go-book-2e/formatter"
  "github.com/shopspring/decimal"
)

• First use go get to get the modules(automatically updates go.mod file) and then build the application.
• Inspect the go.mod file notice, how formatter library has a pseudoversion to formatter package and decimal has a proper version. These both will direct dependencies, rest all will be indirect.
• For each dependency while updating go.mod file, another go.sum file is created containing checksum of each dependency.
• To download a specific library instead of scanning your source code, you can use go get <module_path/url>, in this case all your such dependencies will be in the indirect block. To fix this use go mod tidy

Working with Versions

// go get ./...
go: downloading github.com/learning-go-book-2e/simpletax v1.1.0
go: added github.com/learning-go-book-2e/simpletax v1.1.0
go: added github.com/shopspring/decimal v1.3.1

go build

Now your go.mod file would be

module github.com/learning-go-book-2e/region_tax

go 1.20

require (
    github.com/learning-go-book-2e/simpletax v1.1.0
    github.com/shopspring/decimal v1.3.1
)

• by default go picks latest version of module, to check use go list -m -versions <module_path>
• Now use go get github.com/learning-go-book-2e/[email protected] update module version and check go.mod file now. You will notice now go.sum will contain both version, which is not an issue.
• The version numbers attached to Go modules follow rules of semantic versioning (SemVer) SemVer Specs

Minimal Version Selection

• In case of multiple modules depending on the different version of same module, Go’s module system relies on the principle of minimal version selection.
• It always selects minimum version that would work with all the dependent modules.

Updating to Compatible Versions

• Assume that there is a package simpletax with v1.1.0 version which has a bug patched in v1.1.1 then it was updated with a new feature v1.2.0.
• to explicitly upgrade a depedency, to upgrade to a bug patch release for current minor version use go get -u=patch github.com/learning-go-book-2e/simpletax
• Assume you are on v1.0.0 and run above patch command, then it doesn’t fetch anythings, but on v1.1.0 it fetches the latest patch.
• To get the latest version of simpletax use go get -u github.com/learning-go-book-2e/simpletax

Updating to Incompatible Versions

• Assume that now simpletax features a cross country calculations but API has changed significantly, causing us to use v2.0.0 as per semantic versioning.
• To handle this incompatibility, Go module follow the semantic import versioning rules
  • The major version of the module must be incremented
  • for all major version besides 0 and 1, the path to the module must end with vN where N is major version
• The path changes because an import path uniquely identifies a package. By definition, incompatible versions of a package are not the same package.
• Import becomes : "github.com/learning-go-book-2e/simpletax/v2"

Vendoring

• It refers to practice of keeping the builds consistent with identical dependencies, we keep copies of dependencies inside the module. It could be enable using go mod vendor.
• It creates a vendor directory on top level containing all dependencies

Using pkg.go.dev

• it is a single service that gathers documentation on Go modules.
• It automatically indexes open source Go modules.
• Visit pkg.go.dev

Publishing your Modules

• whenever a module is released as open source on a public VCS like Github or privately hosted version control, it is published.
• There is no need to explicitly upload your module to some central library repo, as for maven/npm we do.
• When releasing an open source module, make sure to include LICENSCE in the root of your repo. Choosing a Good Licensce

Versioning Your Module

• always properly version your modules whether they are public or private to avoid issues with Go’s module systems.
• Always follow proper semantic versioning
• Go supports concept of pre-releases. Let’s assume that the curretn version of your module is tagged v1.3.4. You are working on version 1.4.0, which is not quite done, but to test it you need to publish it. Just add v1.4.0-beta1 to indictate is not quite done or v1.4.0-rc2
• Only when breaking backward compatibility we need to bump the major version number.
• There are two ways to store new version, create a subdirectory within your module named nN, where N is the major version of your module or create a branch in VCS, you can put either the old code or new code on the new branch. Name the branch vN
• Its a good idea to tag the repo in case you are using the same subdirectory system or keeping the code in the main branch only.
• Go Modules: v2 and Beyond and Developing a major version update

Overriding Dependencies

• whenever a project is unmaintained, people often create fork of that project and you would like to get the community driven updates to fork by overriding the default package.

• go replace github.com/jonbodner/proteus => github.com/someone/my_proteus v1.0.0

• NOTE: A replace directive can also refer to a path in your local file system as well (in this case version must be omitted).

• To block a specific version of a module from being used.

• go exclude github.com/jonbodner/proteus v0.10.1

Retracting a Version of your Module

• If you accidentally published some buggy or wrong version of your code, you can retract your version by using retract directive in go.

retract v1.5.0 // not fully tested
retract [v1.7.0, v.1.8.5] // posts your cat photos to LinkedIn w/o permission

• these directive in go.mod requires to create a new version of your module.
• When a version is retracted, existing builds that specified the version will continue to work, but go get and go mod tidy will not upgrade to them. They will no longer appear as options when you use the go listcommand. If the most recent version of a module is retracted, it will no longer be matched with @latest; the highest unretracted version will match instead.

Using Workspaces to Modify Modules Simultaneously

• Go workspaces allows users to have multiple module downloads and references between those modules will automatically resolve to local source code instead of the code hosted in your repository.

go work init ./workspace_app  // creating workspace
go work use ./workspace_lib // use local copy of workspace_lib package

• above creates, go.work file. This file is supposed for local development only, do not commit to repo

// go.work

go 1.20

use (
    ./workspace_app
    ./workspace_lib
)

cd workspace_app
go build
./workspace_app // works as expected.

// since workspace_lib works as expected, it can be pushed to github.

git init
git add .
git commit -m "first commit"
git remote add origin [email protected]:learning-go-book-2e/workspace_lib.git
git branch -M main
git push -u origin main

• Now all your previously not working commands like go get ./... etc will start working

Module Proxy Servers

• Every module is stored in a source code repo, like Github or GitLab. But by default go get doesn’t ffetch code directly instead sends request to a proxy server run by Google which server as a cache store.
• Google also maintains the checksum database storing information about every module ever cached by thier server.

Specifying a Proxy Server

• you can disable proxying by setting GOPROXY=direct in environment variables.
• you can run your own proxy server as well. Both Artifactory and Sonatype have their own Go proxy server support built-in in their enterprise plans.

Using Private Repositories

• To use private repositories you have to run your private proxy server, so that you don’t request it from Google or disable proxy.
• If using a public proxy server we can set GOPRIVATE=*.example.com, company.com/repo

Complete Go Modules Guide