Introduction to Buck

31 October 2013

Shawn Pearce

Maintainer, Gerrit Code Review

Overview

tl;dw: Buck is fast, Maven is ... not

Outline

What is Buck?

java_library(
  name = 'greet_api',
  srcs = ['Greeter.java', 'Group.java'],
)
                      
java_library(
  name = 'hello_world',
  srcs = ['HelloGreeter.java', 'WorldGroup.java'],
  deps = [':greet_api'],
)

Maven vs. Buck

mvn package -Dmaven.{javadoc,test}.skip=true  ... 6m50s
buck build :gerrit                            ... 2m 3s
buck test --all                               ... 2m 5s

mvn package -Dmaven.{javadoc,test}.skip=true  ... 4m44s
buck build :gerrit                            ...    2s

Introduction to Buck

Installation

Buck comes in source form as a Git repository

git clone https://github.com/facebook/buck
cd buck
ant                # 8.736s

PATH=$(pwd)/bin:$PATH

Alternative is to symlink to buck

ln -s $(pwd)/bin/buck  ~/bin/buck
ln -s $(pwd)/bin/buckd ~/bin/buckd       # optional

Android SDK configuration is only necessary to build Android applications

Build Rules

Written in files named BUCK

java_library(
  name = 'client',
  srcs = ['Client.java', 'Window.java'],
)

Build Rules - Naming

Targets (rules) are identified by filesystem path and name

plugins/replication/BUCK:
  java_library(name = 'replication', ...)
  java_library(name = 'utils', ...)
is referred to as
//plugins/replication:replication

Building Java Stuff

Example Java Project

lib/BUCK

prebuilt_jar(
  name = 'guava',
  binary_jar = 'guava.jar', 
)

src/BUCK

java_library(
  name = 'client',
  srcs = ['Client.java', 'Window.java'],
  resources = ['Client_en.properties', 'icon.gif'],
  deps = ['//lib:guava'],
)

java_binary(
  name = 'my_app',
  main_class = 'Client',
  deps = [':client'],
)

Java Rules - Libraries

prebuilt_jar(
  name = 'guava',
  binary_jar = 'guava.jar', 
)
java_library(
  name = 'client',
  srcs = ['Client.java', 'Window.java'],
  resources = ['Client_en.properties', 'icon.gif'],
  deps = ['//lib:guava'],
)

Java Rules - Application Binary

java_binary(
  name = 'my_app',
  main_class = 'Client',
  deps = [':client'],
)

Java Rules - Testing

java_test(
  name = 'tests',
  srcs = ['ClientTest.java', 'WindowTest.java'],
  deps = [':client', '//lib:junit'],
  source_under_test = [':client'],
)
java_test(name = 'ClientTest', srcs = ['ClientTest.java'], deps = [':client', '//lib:junit'])
java_test(name = 'WindowTest', srcs = ['WindowTest.java'], deps = [':client', '//lib:junit'])

Macro Rules

Python functions to shorten build files

def parallel_test(srcs, deps = []):
  """Defines one java_test rule per source file."""
  for s in srcs:
    n = s[:s.index('.java')]
    java_test(name = n, srcs = [s], deps = deps)

Use macros in BUCK files

include_defs('//tools/parallel_test.defs')

parallel_test(
  srcs = glob(['*Test.java']),
  deps = [':client', ':junit'],
)

Special Function glob()

TEST = glob(['src/main/java/**/*Test.java'])
API = ['src/main/java/com/mypkg/Greeter.java',
       'src/main/java/com/mypkg/Group.java']

java_library(
  name = 'hello_world',
  srcs = glob(['src/main/java/**/*.java'], excludes = API + TEST),
  deps = [':api'],
)

java_library(name = 'api', srcs = API)

java_test(
  name = 'tests',
  srcs = TEST,
  deps = [':api', ':hello_world'],
)

Not Compiling Java? - genrule()

Automatically download from Maven Central

prebuilt_jar(
  name = 'some_lib',
  binary_jar = genfile('some.jar'),
  deps = [':download'],
)

genrule(
  name = 'download',
  cmd = 'curl http://repo1.maven.org/maven2/.../some.jar >$OUT',
  out = 'some.jar',
)

Custom Logic? - genrule()

Concatenate 3 files together

genrule(
  name = 'concat',
  srcs = ['content.txt', 'header.html', 'footer.html'],
  cmd = 'cat $SRCDIR/header.html $SRCDIR/content.txt $SRCDIR/footer.html >$OUT',
  out = 'content.html',
)

ANTLR Support? - genrule()

genrule(
  name = 'gen_antlr',
  srcs = ['Query.g'],
  cmd = '$(exe :antlr) -o $TMP $SRCS; jar cf $OUT -C $TMP .',
  deps = [':antlr'],
  out = 'query.src.zip',
)

java_library(
  name = 'query_parser',
  srcs = [genfile('query.src.zip')],
  deps = [':gen_antlr'],
)

prebuilt_jar(name = 'antlr_jar', binary_jar = 'antlr.jar')
java_binary(name = 'antlr', main_class = 'org.antlr.Tool', deps = [':antlr_jar'])

Gerrit Code Review's Maven Build

What is Gerrit Code Review?

Maven build process

Migration to Buck - Parallel Builds

java_library(
  name = 'this-module',
  srcs = glob(['src/main/java/**/*.java']),
  resources = glob(['src/main/resources/**/*']),
  deps = [
    '//other-module:other-module',

    '//lib:gson',
    '//lib:gauva',
  ],
  visibility = ['PUBLIC'],
)

maven_jar Macro

lib/BUCK

include_defs('//lib/maven.defs')

maven_jar(
  name = 'gson',
  id = 'com.google.code.gson:gson:2.1',
  sha1 = '2e66da15851f9f5b5079228f856c2f090ba98c38',
  license = 'Apache2.0',
)

maven_jar(
  name = 'guava',
  id = 'com.google.guava:guava:14.0',
  sha1 = '67b7be4ee7ba48e4828a42d6d5069761186d4a53',
  license = 'Apache2.0',
)

Getting Along with Buck

Successful Buck Tips #1 - Buck Version

Create .buckversion with Git SHA-1 of Buck:

(cd ~/buck; git rev-parse HEAD) >.buckversion
git add .buckversion

Temporarily disable version enforcement using .nobuckcheck

touch .nobuckcheck

Successful Buck Tips #2 - Project Defaults

Add aliases for common targets to .buckconfig

[alias]
  docs = //Documentation:html
  server = //gerrit-server:server

buck build docs          # instead of buck build Documentation:html

Configure project-wide macro rules in .buckconfig

[buildfile]
  includes = //tools/default.rules

Successful Buck Tips #3 - Even Faster Builds

Use buckd for faster incremental builds:

~/buck/bin/buckd
time buck :gerrit      ...  0.5s        # without buckd, 2s

If using git and buckd, ignore .git in .buckconfig

[project]
  ignore = .git

Enable local artifact caching in .buckconfig.local

[cache]
  mode = dir
  dir = buck-cache

Successful Buck Tips #4 - Maven Converts

Configure source roots in .buckconfig

[java]
  src_roots = java, resources

Borrow Gerrit Code Review's maven_jar() rule

maven_jar(
  name = 'guava',
  id = 'com.google.guava:guava:15.0',
  sha1 = 'ed727a8d9f247e2050281cb083f1c77b09dcb5cd',
  license = 'Apache2.0',
)

Thank you

Shawn Pearce

Maintainer, Gerrit Code Review