kotlinx-knit

by Kotlin

Kotlin source code documentation management tool

169 Stars 5 Forks Last release: 11 days ago (0.2.2) Other 55 Commits 9 Releases

Available items

No Items, yet!

The developer of this repository has not created any items for sale yet. Need a bug fixed? Help with integration? A different license? Create a request here:

Knit tool

JetBrains incubator project Apache license Download

Kotlin source code documentation management tool.

This is a tool that produces Kotlin source example files and tests from markdown documents with embedded snippets of Kotlin code. It also helps to add links to the API documentation website into the documents and has a few other helpful markdown-management features.

Knit tool is a Gradle plugin that processes markdown (

.md
) files and Kotlin (
.kt
/
.kts
) files with markdown KDoc comments, updates them, and writes additional example and test files, which are committed to the VCS. The overall workflow is:
  1. Write or update documentation in markdown files (
    .md
    ) or in source files (
    .kt
    /
    .kts
    ).
  2. Run
    gradlew knit
    to update markdown and source files, generate additional source code samples and tests.
  3. Commit to VCS.
  4. Generated files are automatically verified on subsequent project builds.

Knit does not really parse markdown format or HTML, but understands certain Knit markup patterns and directives. Directives in markdown files must always start at the beginning of the line and have the following general format for single-line directives:


or the following format for multi-line directives:


Directives look like HTML comments, so their contents are not visible when the markdown is rendered by regular tools. Specific markup patterns and directives supported by Knit are explained in the Features section. For inclusion of directives into Kotlin source (

.kt
/
.kts
) files see the Kotlin Source Comments section.

Contents

Setup

Knit is a Gradle plugin that is published to JCenter and Maven Central. Add it to the

build.gradle
in the following way:
buildscript {
    dependencies {
        classpath "org.jetbrains.kotlinx:kotlinx-knit:0.2.2"
    }
}

apply plugin: 'kotlinx-knit'

The build must apply 'kotlin' plugin or, at least, 'base' plugin before 'kotlinx-knit'.

Tasks

Knit plugin registers the following tasks:

  • knit
    — updates markdown files, samples, and tests.
  • knitCheck
    — checks that all the files are up-to-date and fail the build if not; it is automatically added as dependency to
    check
    task and thus is performed on
    build
    .
  • knitPrepare
    — does nothing, but is added as a dependency to both
    knit
    and
    knitCheck
    and a common place to register all prerequisite tasks like
    dokka
    (see Dokka setup)

Optional parameters

Additional optional parameters can be specified via

knit { ... }
DSL with the defaults as shown below.
knit {
    rootDir = project.rootDir // project root dir
    // Custom set of input files to process (default as shown below)
    files = fileTree(project.rootDir) {
        include '**/*.md'
        include '**/*.kt'
        include '**/*.kts'
        exclude '**/build/**'
        exclude '**/.gradle/**'
    }
}

Knit properties

Some Knit features use additional properties. These properties are stored in

knit.properties
file that is located in the same directory as the corresponding markdown file. Knit tool also looks for the properties file in all the parent directories up to the root directory (see Optional parameters). This allows for fine-grained control and inheritance of properties in different parts of the project.

All paths that are specified in the property files are resolved relative to the directory of the corresponding property file. Paths specified in markdown files are relative to the corresponding markdown file, too.

Features

All Knit features are driven by feature-specific patterns and directives and can be used independently.

Example files

Knit can generate Kotlin source examples from the code that is being quoted in the documentation. To set it up you need to specify at least the following two properties in

knit.properties
, for example:

knit.dir=src/test/kotlin/example/
knit.package=com.example

The

knit.dir
must specify the relative path to the directory for the examples (note that it must end with
/
) and
knit.package
must specify the package name for the example files. The directory is usually marked as or located inside the project's test sources and gets compiled when the project is built. This way, Knit tool helps to ensure that all the code in the documentation is syntactically correct and compiler without errors.

In the markdown file Knit collects together all the

kotlin
sources in the markdown that are surrounded by a triple backticks like this:
```kotlin
fun foo() {}
```

The Knit que to generate example source code is a markdown reference in round braces to the file that needs to be generated. It must start with the value of

knit.dir
property (verbatim) followed by the example's file name, for example:
> You can get the full code [here](src/test/kotlin/example/example-basic-01.kt).

The name of the example file must match a specific pattern. By default, this pattern's regex is

example-[a-zA-Z0-9-]+-##\\.kt
. It can be overridden via
knit.pattern
property. The sequence of hashes (
#
) in the pattern matches any alpha-numeric sequence and causes the examples to be automatically consecutively numbered inside the markdown file. For example, you can add a new section of code at the beginning of the document and write in the markdown file:
> You can get the full code [here](src/test/kotlin/example/example-basic-new.kt).

After running

knit
task this line in the markdown file will get updated to:
> You can get the full code [here](esrc/test/kotlin/example/example-basic-01.kt).

The corresponding Kotlin file is also automatically created or updated as needed by

knit
task and will look like this:
// This file was automatically generated from example-basic.md by Knit tool. Do not edit.
package com.example.exampleBasic01

fun foo() {}

Explicit knit directive

A

KNIT
directive can be used to trigger generation of an example file instead of providing a readable reference to a file. In this case, only an example file name shall be specified (without a path). An example file will be written to the
knit.dir
property directory. The name of the file shall still match the
knit.pattern
property pattern and it will be automatically numbered as explained in the previous section. For example:
```kotlin
fun foo() {}
```

 

Merging code pieces

All tripple-backquoted Kotlin sections are merged together and are output to the Kotlin source file when the next Knit pattern in encountered. This way, documentation can be written fluently, explaining functions as they are introduced. For example, the following markdown:

This function computes the square of the given integer:

fun sqr(x: Int) = x * x

We can use to print the square:

fun main() {
    println(sqr(5))
}

> You can get the full code here.

Produces the following Kotlin source code when

knit
task is run:
// This file was automatically generated from example-merge.md by Knit tool. Do not edit.
package com.example.exampleMerge01

fun sqr(x: Int) = x * x

fun main() { println(sqr(5)) }

Custom Knit template

The header of this generated example file can be configured by specifying

knit.include
property that contains a path to the FreeMarker template file. The default template is:
// This file was automatically generated from ${file.name} by Knit tool. Do not edit.
package ${knit.package}.${knit.name}

Each example file gets its unique package name where

${knit.package}
part is taken from properties and
${knit.name}
part is automatically generated by camel-casing the example file name. This allows to have a number of similar examples in the same markdown file that might, for example, define different versions of the function with the same name so that they could still be compiled, because they end up in different packages.

You can use arbitrary

knit.xxx
properties in the template and introduce your own properties so that you can reuse the same template in multiple kinds of documents in your project. The
knit.package
property is not special in any way.

Include directive

Sometimes it is necessarily to define example-specific additions (like Kotlin

import
lines) that should not be visible to the readers of documentation to avoid distraction. For example, the documentation for Kotlin's
kotlin.system.exitProcess
function might have the following example piece of code:
fun exit(): Nothing = exitProcess(0)

It will not compile by itself. In order to generate a proper compilable example file we'd use an

INCLUDE
Knit directive before this example. The markdown documentation looks like this:
fun exit(): Nothing = exitProcess(0)

> You can get the full code here.

The Knit directive is like HTML comment, so the reader of this specific piece of documentation will not see the

import
line, but the generated source-code example file will include it to get compiled properly:
// This file was automatically generated from example-include.md by Knit tool. Do not edit.
package com.example.exampleInclude01

import kotlin.system.*

fun exit(): Nothing = exitProcess(0)

Advanced include

A single piece of code can be included into multiple examples (as opposed to the next example only) by specifying regex pattern of the example file name right after the

INCLUDE
directive as its parameter.

With the pattern the

INCLUDE
directive can also be specified on a single line, without the code inside of it. In this case, the code to be included is taken from the previously tripple-backquoted Kotlin source code before it. This way, the code snippet can be introduced and shown to the reader of the documentation and then included into the several subsequent examples.

Prefix directive

If you need to prefix the example file with certain lines, for example, to specify a file-level annotation, then write the block of code normally and follow it with the

PREFIX
directive. All the previously written code will be added to the beginning of the resulting example file instead of the end of it. So this input:
```kotlin 
@file:JvmName("ExampleKt")
```      



fun example() {}

> You can get the full code here.

Produces this example file:

@file:JvmName("ExampleKt")
// This file was automatically generated from example-prefix.in.md by Knit tool. Do not edit.
package com.example.examplePrefix01

fun example() {}

Just like the

INCLUDE
directive you can put the code inside the
PREFIX
directive, so that the reader does not see it (because it is a comment), but it affects the resulting example file. A parameter with the file name pattern can be also added to the
PREFIX
directive to affect multiple examples.

Suffix and back-to-back directives

The

SUFFIX
directive queues a piece of code to be added to the end of the example file and supports all the same features as
INCLUDE
and
PREFIX
, including patterns. This way, using a combination of
INCLUDE
and
SUFFIX
, a specific common scaffolding can be defined around several code examples matching a specific name pattern. When multiple directives are written back-to-back they can be laid out as a single HTML comment, using
-----
to end one directive and begin the next one.

The following expression:

"OW".replace('W', 'K')

> You can get the full code here.

Tests

Knit tool can also automatically generate tests. To set it up you to be generating example files first and then add the following properties in

knit.properties
:

test.dir=src/test/kotlin/example/test/
test.package=com.example.test

Here

test.dir
specified the directory where the Kotlin test code is generated too and
test.package
specifies the package. In the beginning of the markdown file you specify the name of the test using
TEST_NAME
directive. There is one test file per the source markdown file with a test-case for each example. After the example you can place the expected output of the test in tripple-quoted
text
block and add
TEST
directive after it to get the test-case added. For example:
 

Here is some explanatory text

fun main() {
    println("Hello, world!")
}

> You can get the full code here.

This code prints:

Hello, world!

Based on these directives, the

knit
task will create
BasicTest.kt
file with the following contents:
// This file was automatically generated from test-basic.md by Knit tool. Do not edit.
package com.example.test

import org.junit.Test import kotlinx.knit.test.*

class BasicTest { @Test fun testExampleBasic01() { captureOutput("ExampleBasic01") { com.example.exampleBasic01.main() }.verifyOutputLines( "Hello, world!" ) } }

The test runs the generated example, assuming it defines

main
function, and verifies the produced output. Two helper functions
captureOutput
and
verifyOutputLines
are provided in a separate artifact that you need to add to your test dependencies to compile and run the resulting test:
dependencies {
    testImplementation "org.jetbrains.kotlinx:kotlinx-knit-test"
}

You don't need this dependency if you use a custom test template that is using your project-specific functions.

Hidden test

If you do not want to include the sample output in the documentation itself, but still want the test to be generated, then you can include the expected output into the

TEST
directive itself, for example:

Custom test predicate

If the output of the sample code can be non-deterministic you'd need to write test verification logic. If this logic is single-liner, then you can specify the corresponding test predicate directly as parameter to

TEST
directive operating over
lines: List
, for example, in order to check that the example had output an integer between 1 and 100 you can write:
   

Output comparison mode

A different output comparison mode can be specified by adding the name of the mode after the

TEST
directive. The names of the modes are mapped to comparison functions via
test.mode.
properties, with
verifyOutputLines
function being used as a default. Other modes that are supported out of the box:
  • LINES_START
    — tests that output starts with the specified lines, skipping the rest of line, uses
    verifyOutputLinesStart
    function.

Test template

Generation of the test source code is completely template-based. The default template is located in

knit.test.template
file and can be overridden via

test.template
property. You can use arbitrary
test.xxx
properties in the test template.

The default template assumes that example code contains

main()
function and produces some output on the console. By tweaking the template you can test other kinds of examples in your markdown documentation.

Kotlin Source Comments

Knit directives and other Knit-recognized markdown markup can be embedded into documentation of Kotlin source (

.kt
/
.kts
) files. There are several ways to embed Knit markup into Kotlin sources.

Knit markup can be nested inside regular /* ... */ comment block, directives starting at the beginning of the line. For example:

/* Include the following snippet into all generated examples

*/

Knit markup can be specified after

//
line comment, directives separated by one space from the beginning of the comment. The whitespace character after the
//
start marker is dropped when reading the directive body.
// Prefix the following example with this annotation
// 

Knit markup can be specified inside

/** ... */
KDoc comments, separated by one space from the
*
at the beginning of comment lines. The Knit tool does not really parse Kotlin files. It just looks at the
*
character at the beginning of the line.
/**
 * The ultimate answer to life, universe, and everything can be printed like this:
 * ```kotlin
 * fun main() {
 *     println(theAnswer())
 * }
 * ```
 * 
 */
 fun theAnswer() = 42        

API references

Knit tool can add links to project's API documentation, so that you can link to the public classes and functions similarly to how you do it from KDoc using markdown

[name]
reference syntax.

This feature is not available inside Kotlin source (

.kt
/
.kts
) files, because API documentation references inside KDoc comments are processed by the Dokka tool.

Dokka setup

In order to generate links to project's API documentation this documentation must be built using Dokka in either

markdown
,
jekyll
, or
html
formats:
dokka {
    outputFormat = "jekyll" 
    outputDirectory = "$buildDir/dokka"
}

Website's root for Knit must be configured as shown below:

knit {          
    // Required parameter
    siteRoot = "https://example.com"  // website with project's API documentation without trailing /
    // Optional parameters (do not need specify them if below defaults are Ok) 
    moduleRoots = ["."] // list directories that contain project modules (subdir name == module name)
    moduleMarkers = ["build.gradle", "build.gradle.kts"] // marker files that distinguish module directories
    moduleDocs = "build/dokka" // where documentation is build into relative to module root 
}                       

// Build API docs for all modules with dokka before running Knit knitPrepare.dependsOn rootProject.getTasksByName("dokka", true)

The modules providing APIs can be stored in separate directories named after the module name. For example, this project has

kotlinx-knit-test
module in a separate directory. You can reference functions and classes declared there using a regular markdown link syntax and give instructions to Knit tool to expand those links like this:

Here is a link to [captureOutput] function.




The

MODULE
directive specified the name of the module. Knit looks for the corresponding directory that contains one of the configured
moduleMarkers
files. This directive is followed by one or more
INDEX
directives that specify package names.

When you run

knit
task this markdown gets updated to:
Here is a link to [captureOutput] function.




Now the link is defined to point to

///
.

When the documentation is build by the root project of the module, then prepend

/
before the module name in the
MODULE
directive:

Table of contents

Knit can generate "Table of contents" for big markdown file that includes references to all second-level and smaller-level header. Just put

TOC
and
END
directives at the beginning of the markdown file like this:

On the next run of

knit
task the table of contents will get placed in between them, replacing all the text previously contained there. This README file is an example markdown file using this feature. See Contents section in the beginning.

We use cookies. If you continue to browse the site, you agree to the use of cookies. For more information on our use of cookies please see our Privacy Policy.