Creating git version history for composer packages

by Wasalu Duckworth

First thing you may be wondering why this is necessary? Composer packages are sometimes based on projects where history or change log is not publicly available. Or perhaps you do not trust the change log to show the full picture. Or perhaps you need to know what exactly changed in which version or perhaps find the context of why some adjustments were implemented. The uses cases are unlimited.

The usual solution in this scenario is to install each of the versions you are interested and do a manual diff. This is not difficult (no pun intended) but is not very optimal in case you are looking for something unknown or need to get a bigger picture view across many versions.

I was recently in a similar problem therefore I decided to create a simple tool that would generate the diff for each version in a easy to manage and easy to browse type of way. What is the easiest way to keep and see differences between versions? Hmm...

Obviously it is version control and since my favorite tool is git there was no doubt on which software to use for version control.

The next consideration was on how to write the script. Simple bash script? Could but does not seem very fun. Perhaps we should base this on composer itself and somehow load it as a package? Then we could use the tool to generate the diff for itself, inception. Sounds better but still does not seem fun. Instead I decided to experiment with a language that has been getting my interest for a while - golang. We can easily create software that can be installed on various operating systems and the language is preached as best of all worlds.

Now that technology has been decided upon lets set the plan for what we are going to create. Here is how it will go:

  • User inputs a package name
  • The program fetches the package via composer and generates a commit for each version.
  • In this process a history of each version is stored in git which gives us great flexibility to view the changes across versions.

Now lets get to the implementation. Lets use Magento composer helper as an example package we want to process.

Normally you would install the package using:

composer require magento/composer

So the idea is clear we need to allow the user to input the package name. 

func main() {

	var packageName string
	fmt.Print("Enter composer package:\n")

	fmt.Scanln(&packageName)
    
    
    fmt.Printf("Composer package: %s\n",packageName)
}

Ok, we now have a program that allows to receive user input. Great. The next step is to figure out how we are going to get the all version for the package. We can of course install a specific package by using a command like this. 

composer require "drupal/admin_toolbar:1.15"

However since we do not care about dependencies we should make sure there are not errors when some requirements are not met. Therefore we should also use the --ingore-platfrom-reqs flag. Translating this to golang we get the following function.

func installPackageVersion(packageName,version string) {
	success, error := exec.Command("composer", "require",packageName + ":" + version,"--ignore-platform-reqs","-W").Output()
	if error != nil {
		fmt.Printf("Version %s for %s could not be installed because of %s\n", version, packageName, error)
	} else {
		fmt.Print(success)
	}
}

In the function you can notice a few things. First we are calling the exec.Command to execute the composer command. This means that installed composer will be a requirement for our software to work. For now this will suffice as we assume the program will be used only by developers for whom this is already installed. Long term plan would be to make the software without any dependencies but that is out of scope for this article.

Great. Now we can get the user input and fetch a certain version for this package. However what we really want to do is to get all versions of the package. Unfortunately there is no simple way to simply list all version number however we can get this information by using the following composer command:

composer show magento/composer --all --format=json

The composer show command with the --all flag returns information about the package, including a node versions which contains all package versions. Notice we are also using the --format=json flag to make sure the data is returned as json. 

	packageData, error := exec.Command("composer","show", packageName, "--all", "--format=json").Output()
	if error != nil {
		return nil
	} else {
		return getPackageVersions(packageData)
	}

Now that we have the package information as json the next step would be to parse the json data and get a list of versions so that we can process each of them. In golang we can parse the json content by using json.Unmarshal. We need to pass in our byte array to the Unmarshal function which will transform it into a map of string interfaces. We can then use this to loop the versions node and fetch the information we need.

func getPackageVersions(packageData []byte) []string{
	var raw map[string]interface{}
	if err := json.Unmarshal(packageData, &raw); err != nil {
		panic(err)
	}
	var versionList []string
	for k, v := range raw {
		switch vv := v.(type) {
			case string:
			case int:
			case []interface{}:
				if k == "versions" {
					for _, u := range vv {
						version := fmt.Sprint(u)
						versionList = append(versionList, version)
					}
				}
			default:
		}
	}
	return versionList
}

We are using a switch case statement which allows us to safely loop the json range. This is the suggested way to process json in golang so we do not have to worry about any nodes changing their type.

Now that we have collected the available versions and have a command that allows to install each of them we need to take care of the version control part. Here the idea is simple - we need to create a directory where we will initialize git and commit each of the versions. First lets create a directory and move our script into it.

func createAndMoveToDirectory(directoryName string) bool {
	directoryName = normalizeString(directoryName)
	cmd := exec.Command("mkdir", directoryName )
	err := cmd.Run()
	if err != nil {
		fmt.Printf("Directory %s could not be created because of %s.\n", directoryName, err)
		return false
	}

	fmt.Printf("Directory %s created successfully.\n", directoryName)

	if !moveToDirectory(directoryName) {
		return false
	}

	return true
}
func moveToDirectory(directoryName string) bool {
	err := os.Chdir(getCurrentDirectory() + "/" + directoryName)
	if err != nil {
		fmt.Printf("Directory %s could not be changed because of %s.\n", directoryName, err)
		return false
	}
	fmt.Printf("Moved to %s.\n",directoryName)
	return true
}

We want to move into the directory so that the git actions would occur only in this scope. For creating the directory we can invoke the exec command and execute mkdir function. However better way to handle this would be to use the golang os library which contains OS functions. For moving to the directory we are doing this by using os.Chdir.

Ok, now we have the directory. We only need to be able to create commits and everything should work as expected. Lets initialize git.

func initializeGit() bool{
	fmt.Print("Git init\n")
	cmd := exec.Command("git", "init")
	err := cmd.Run()
	if err != nil {
		fmt.Printf("Could not initialize git because - %s.\n", err)
		return false
	}
	fmt.Print("Git init successful.\n")
	return true
}

Next step is to create a function that will allow us to commit the files in each version. We simply need to add all current files to be staged and then create a commit. See the function below.

func createCommit(version string) bool {
	fmt.Print("Add files to commit.\n")
	addFilesCmd := exec.Command("git", "add", ".")
	addErr := addFilesCmd.Run()
	if addErr != nil {
		fmt.Printf("Could not add files to git because - %s.\n", addErr)
		return false
	}

	commitCmd := exec.Command("git", "commit", "-m", "Version: " + version)
	commitErr := commitCmd.Run()
	if commitErr != nil {
		fmt.Printf("Could not commit files to git because - %s.\n", commitErr)
		return false
	}

	fmt.Printf("%s commited successfully.\n",version)
	return true
}

Important to note that we are also adding the commit message indicating the current version. This will allow us to understand and refer to each version while using the history.

Now that we have this we can put all of this together and create the function which will execute the initial plan - enter package name, fetch the versions, create directory, initialize git, loop all versions and install, commit each of them. Here you go (pun intended):

func main() {
	var packageName string
	fmt.Print("Enter composer package:\n")

	fmt.Scanln(&packageName)

	if !createAndMoveToDirectory(packageName) || !initializeGit() {
		return
	}

	availableVersions := getAvailablePackageVersions(packageName)

	for i := len(availableVersions)-1; i >= 0; i-- {				           installPackageVersion(packageName,availableVersions[i])
		createCommit(availableVersions[i])
	}

	fmt.Printf("Successfully created git history for package versions.\n")
	} else {
		fmt.Printf("Package %s is not valid and we can not process it.\n", packageName)
	}
}

After putting this all together we can easily generate the history for any composer package. There are a few issues with this, for example, the packages can be in the wrong order, there is little validation for any errors, we are still dependent on the system to have both git and composer installed and working. In additional our program is still only in development phase and can not be executed as a normal script.

That is what we are going to explore in future articles - finalizing the software and making it work as a standalone app. Let us know if you would like to see the full source code and have any suggestions.