diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/composite.md | 106 | ||||
| -rw-r--r-- | docs/docs.md | 204 | ||||
| -rw-r--r-- | docs/examples.md | 236 | ||||
| -rw-r--r-- | docs/fea2_proposal.md | 150 | ||||
| -rw-r--r-- | docs/feaextensions.md | 559 | ||||
| -rw-r--r-- | docs/feax_future.md | 283 | ||||
| -rw-r--r-- | docs/parameters.md | 179 | ||||
| -rw-r--r-- | docs/scripts.md | 1565 | ||||
| -rw-r--r-- | docs/technical.md | 342 | ||||
| -rw-r--r-- | docs/tests.md | 17 | ||||
| -rw-r--r-- | docs/ufo.md | 159 |
11 files changed, 3800 insertions, 0 deletions
diff --git a/docs/composite.md b/docs/composite.md new file mode 100644 index 0000000..ac6c9b7 --- /dev/null +++ b/docs/composite.md @@ -0,0 +1,106 @@ +# Defining composite glyphs + +A composite glyph is one that is defined in terms of one or more other glyphs. +The composite definition syntax described in this document is a subset of the [GlyphConstruction](https://github.com/typemytype/GlyphConstruction) syntax used by Robofont, but with extensions for additional functionality. +Composites defined in this syntax can be applied to a UFO using the [psfbuildcomp](scripts.md#psfbuildcomp) tool. + +# Overview + +Each composite definition is on a single line and has the format: +``` +<result> = <one or more glyphs> <parameters> # comment +``` +where +- `<result>` is the name of the composite glyph being constructed +- `<one or more glyphs>` represents one or more glyphs used in the construction of the composite glyph, with optional glyph-level parameters described below +- `<parameters>` represents adjustments made to the `<result>` glyph, using the following optional parameters: + - at most one of the two following options: + - `^x,y` (where `x` is the amount added to the left margin and `y` is the amount added to the right margin) + - `^a` (where `a` is the advance width of the resulting glyph) + - `|usv` where `usv` is the 4-, 5- or 6-digit hex Unicode scalar value assigned to the resulting glyph + - `!colordef` (currently ignored by SIL tools) + - `[key1=value1;key2=value2;...]` to add one or more `key=value` pairs (representing SIL-specific properties documented below) to the resulting glyph +- `# comment` is an optional comment (everything from the `#` to the end of the line is ignored) + +In addition, a line that begins with `#` is considered a comment and is ignored (as are blank lines). + +If `[key=value]` properties for the resulting glyph are specified but no `|usv` is specified, then a `|` must be included before the `[`. +This insures that the properties are applied to the resulting composite glyph and not to the last glyph in the composite specification. + +# Examples + +In the following examples, +- `glyph` represents the resulting glyph being constructed +- `base`, `base1`, `base2` represent base glyphs being used in the construction +- `diac`, `diac1`, `diac2` represent diacritic glyphs being used in the construction +- `AP` represents an attachment point (also known as an anchor) + +## glyph = base +``` +Minus = Hyphen +``` +This defines one glyph (`Minus`) in terms of another (`Hyphen`), without having to duplicate the contours used to create the shape. + +## glyph = base1 & base2 +``` +ffi = f & f & i +``` +This construct causes a glyph to be composed by aligning the origin of each successive base with the origin+advancewidth of the previous base. Unless overridden by the `^` parameter, the left sidebearing of the composite is that of the first base, the right sidebearing is that of the last, and the advancewidth of the composite is the sum of the advance widths of all the component base glyphs. [Unsure how this changes for right-to-left scripts] + +## glyph = base + diac@AP +``` +Aacute = A + acute@U +``` +The resulting composite has the APs of the base(s), minus any APs used to attach the diacritics, plus the APs of the diacritics (adjusted for any displacement, as would be the case for stacked diacritics). In this example, glyph `acute` attaches to glyph `A` at AP `U` on `A` (by default using the `_U` AP on `tilde`). The `U` AP from `A` is removed (as is the `_U` AP on the `tilde`) and the `U` AP from `acute` is added. + +Unless overridden by the `^` parameter, the advance width of the resulting composite is that of the base. + +## glyph = base + diac1@AP + diac2@APonpreviousdiac +``` +Ocircumflexacute = O + circumflex@U + acute@U +``` +The acute is positioned according to the `U` AP on the immediately preceding glyph (`circumflex`), not the `U` AP on the base (`O`). + +## glyph = base + diac@anyglyph:anyAP + +The syntax allows you to express diacritic positioning using any arbitrary AP on any arbitrary glyph in the font, for example: +``` +barredOacute = barredO + acute@O:U # not supported +``` +Current SIL tools, however, only support an `anyglyph` that appears earlier in the composite definition, so the above example is **not** supported. + +This syntax, however, makes it possible to override the default behavior of attaching to the immediately preceding glyph, so the following is supported (since the `@O:L` refers to the glyph `O` which appears earlier in the definition): +``` +Ocircumflexdotaccent = O + circumflex@U + dotaccent@O:L +``` +The `@O:L` causes the `dotaccent` diacritic to attach to the base glyph `O` (rather the immediately preceding `circumflex` glyph) using the `L` AP on the glyph `O` and the `_L` AP on the glyph `dotaccent`. + +## glyph = base + diac@AP | usv +``` +Aacute = A + acute@U | 00C1 +``` +USV is always given as four- to six-digit hexadecimal number with no leading "U+" or "0x". + +## glyph = base + diac@AP ^ leftmarginadd,rightmarginadd +``` +itilde = i + tilde@U ^ 50,50 +``` +This adds the values (in design units) to the left and right sidebearings. Note that these values could be negative. + +# SIL Extensions + +SIL extensions are all expressed as property lists (`key=value`) separated by semicolons and enclosed in square brackets: `[key1=value1;key2=value2]`. +- Properties that apply to a glyph being used in the construction of the composite appear after the glyph. +- Properties that apply to the resulting composite glyph appear after `|` (either that of the `|usv` or a single `|` if no `|usv` is present). + +## glyph = base + diac@atAP[with=AP] +``` +Aacute = A + acute@Ucap[with=_U] +``` +The `with` property can be used to override the default AP, \_AP convention. The `_U` attachment point on the `acute` glyph is paired with the `Ucap` attachment point on the + +## glyph = base + diac@AP[shift=x,y] + +Aacute = A + acute@U[shift=100,100] + +By applying the `shift` property to the `acute` glyph, the position of the diacritic relative to the base glyph `A` is changed. diff --git a/docs/docs.md b/docs/docs.md new file mode 100644 index 0000000..0759d7a --- /dev/null +++ b/docs/docs.md @@ -0,0 +1,204 @@ +# Pysilfont - utilities for font development + +Pysilfont is a collection of tools to support font development, with an emphasis on [UFO](#ufo-support-in-pysilfont)-based workflows. + +In addition to the UFO utilities, there is also support for testing using [FTML](#font-test-markup-language) and [Composite Definitions](#composite-definitions). + +Some scripts are written specifically to fit in with the approaches recommended in [Font Development Best Practices](https://silnrsi.github.io/FDBP/en-US/index.html) + +# Documentation + +Documentation is held in the following documents: + +- docs.md: This document - the main document for users +- [scripts.md](scripts.md): User documentation for all command-line tools and other scripts +- [technical.md](technical.md): Technical details for those wanting write scripts or other development tasks +- Other sub-documents, with links from the above + +Installation instructions are in [README.md](../README.md) + +# Scripts and commands +Many Pysilfont scripts are installed to be used as command-line tools, and these are all listed, with usage instructions, in [scripts.md](scripts.md). This also has details of some other example python scripts. + +All scripts work using a standard framework designed to give users a consistent interface across scripts, and common features of these scripts are described in the following sections, so the **documentation below** needs to be read in conjunction with that in [scripts.md](scripts.md). + +## Standard command line options + +Nearly all scripts support these: + +- `-h, --help` + - Basic usage help for the command + - `-h d` + - Display -h info with added info about default values + - `-h p` + - Display information about parameters (-p --params below) +- `-q, --quiet` + - Quiet mode - only display severe errors. See reporting below +- `-l LOG, --log LOG` + - Log file name (if not using the default name). By default logs will go in a logs subdirectory. If just a directory path is given, the log will go in there using the default name. +- `-p PARAMS, --params PARAMS` + - Other parameters - see below + +The individual script documentation in scripts.md should indicate if some don't apply for a particular script + +(There is also a hidden option --nq which overrides -q for use with automated systems like [smith](https://github.com/silnrsi/smith) which run scripts using -q by default) + +# Parameters + +There are many parameters that can be set to change the behaviour of scripts, either on the command line (using -p) or via a config file. + +To set a parameter on the command line, use ``-p <param name>=<param value>``, eg +``` +psfnormalize font.ufo -p scrlevel=w +``` +-p can be used multiple times on a single command. + +Commonly used command-line parameters include: +- scrlevel, loglevel + - Set the screen/logfile level from increasingly verbose options + - E - Errors + - P - Progress (default for scrlevel) + - W - Warnings (default for loglevel) + - I - Information + - V - Verbose +- checkfix (UFOs only) + - Validity tests when opening UFOs. Choice of None, Check, Fix with default Check + - See description of check & fix under [normalization](#normalization) + +For a full list of parameters and how to set them via a config file (or in a UFO font) see [parameters.md](parameters.md). + + +## Default values + +Most scripts have defaults for file names and other arguments - except for the main file the script is running against. + +### Font/file name defaults + +Once the initial input file (eg input font) has been given, most other font and file names will have defaults based on those. + +This applies to other input font names, output font names, input file names and output file names and is done to minimise retyping repeated information like the path the files reside in. For example, simply using: + +``` +psfsetpsnames path/font.ufo +``` + +will: + +- open (and update) `path/font.ufo` +- backup the font to `path/backups/font.ufo.nnn~` +- read its input from `path/font_psnames.csv` +- write its log to `path/logs/font_psnames.log` + +If only part of a file name is supplied, other parts will default. So if only "test" is supplied for the output font name, the font would be output to `path/test.ufo`. + +If a full file name is supplied, but no path, the current working directory will be used, so if “test.ufo” is supplied it won’t have `path/` added. + +### Other defaults + +Other parameters will just have standard default values. + +### Displaying defaults for a command + +Use `-h d` to see what defaults are for a given command. For example, + +``` +psfsetpsnames -h d +``` + +will output its help text with the following appended: + +``` +Defaults for parameters/options + + Font/file names + -i _PSnames.csv + -l _PSnames.log +``` + +If the default value starts with “\_” (as with \_PSnames.csv above) then the input file name will be prepended to the default value; otherwise just the default value will be used. + +## Reporting +Most scripts support standardised reporting (logging), both to screen and a log file, with different levels of reporting available. Levels are set for via loglevel and scrlevel parameters which can be set to one of: +- E Errors +- P Progress - Reports basic progress messages and all errors +- W Warning - As P but with warning messages as well +- I Info - As W but with information messages as well +- V Verbose - even more messages! + +For most scripts these default to W for loglevel and P for scrlevel and can be set using -p (eg to set screen reporting to verbose use -p scrlevel=v). + +-q --quiet sets quiet mode where all normal screen messages are suppressed. However, if there are any errors during the script execution, a single message is output on completion listing the counts for errors and warnings. + +## Backups for fonts + +If the output font name is the same as the input font name (which is the default behaviour for most scripts), then a backup is made original font prior to updating it. + +By default, the last 5 copies of backups are kept in a sub-directory called “backups”. These defaults can be changed using the following parameters: + +- `backup` - if set to 0, no backups are done +- `backupdir` - alternative directory for backups +- `backupkeep` - number of backups to keep + +# UFO support in Pysilfont +With some limitations, all UFO scripts in Pysilfont should work with UFO2 or UFO3 source files - and can convert from one format to the other. + +In addition most scripts will output in a normalized form, designed to work with source control systems. Most aspects of the normalization can be set by parameters, so projects are not forced to use Pysilfont’s default normalization. + +The simplest script is psfnormalize, which will normalize a UFO (and optionally convert between UFO 2 and UFO3 if -v is used to specify the alternative version) + +Note that other scripts also normalize, so psfnormalize is usually only needed after fonts have been processed by external font tools. + +## Normalization +By default scripts normalize the UFOs and also run various check & fix tests to ensure the validity of the UFO metadata. + +Default normalization behaviours include: +- XML formatting + - Use 2 spaces as indents + - Don’t indent the ``<dict>`` for plists + - Sort all ``<dict>``s in ascending key order + - Where values can be “integer or float”, store integer values as ``<integer>`` + - Limit ``<real>`` limit decimal precision to 6 + - For attributes identified as numeric, limit decimal precision to 6 +- glif file names - use the UFO 3 suggested algorithm, even for UFO 2 fonts +- order glif elements and attributes in the order they are described in the UFO spec + +Most of the above can be overridden by [parameters](#parameters) + +The check & fix tests are based on [Font Development Best Practices](https://silnrsi.github.io/FDBP/en-US/index.html) and include: +- fontinfo.plist + - Required fields + - Fields to be deleted + - Fields to constructed from other fields + - Specific recommended values for some fields +- lib.plist + - Required fields + - Recommended values + - Fields that should not be present + +The check & fix behaviour can be controlled by [parameters](#parameters), currently just the checkfix parameter which defaults to 'check' (just report what is wrong), but can be set to 'fix' to fix what it can, or none for no checking. + +## Known limitations +The following are known limitations that will be addressed in the future: +- UFO 3 specific folders (data and images) are preserved, even if present in a UFO 2 font. +- Converting from UFO 3 to UFO 2 only handles data that has a place in UFO 2, but does include converting UFO 3 anchors to the standard way of handling them in UFO 2 +- If a project uses non-standard files within the UFO folder, they are deleted + +# Font Test Markup Language + +Font Test Markup Language (FTML) is a file format for specifying the content and structure of font test data. It is designed to support complex test data, such as strings with specific language tags or data that should presented with certain font features activated. It also allows for indication of what portions of test data are in focus and which are only present to provide context. + +FTML is described in the [FTML github project](https://github.com/silnrsi/ftml). + +Pysilfont includes some python scripts for working with FTML, and a python library, [ftml.py](technical.md#ftml.py), so that new scripts can be developed to read and write FTML files. + +# Composite definitions + +Pysilfont includes tools for automatically adding composite glyphs to fonts. The syntax used for composite definitions is a subset of that used by RoboFont plus some extensions - see [Composite Tools](https://silnrsi.github.io/FDBP/en-US/Composite_Tools.html) in the Font Development Best Practices documentation for more details. + +The current tools (psfbuildcomp, psfcomp2xml and psfxml2comp) are documented in [scripts.md](scripts.md). + +The tools are based on a python module, [comp.py](technical.md#comppy). + +# Contributing to the project + +Pysilfont is developed and maintained by SIL International’s [Writing Systems Technology team ](https://software.sil.org/wstech/), though contributions from anyone are welcome. Pysilfont is copyright (c) 2014-2017 [SIL International](https://www.sil.org) and licensed under the [MIT license](https://en.wikipedia.org/wiki/MIT_License). The project is hosted at [https://github.com/silnrsi/pysilfont](https://github.com/silnrsi/pysilfont). diff --git a/docs/examples.md b/docs/examples.md new file mode 100644 index 0000000..de5212b --- /dev/null +++ b/docs/examples.md @@ -0,0 +1,236 @@ +# Pysilfont example scripts + +In addition to the main pysilfont [scripts](scripts.md), there are many further scripts under pysilfont/examples and its sub-directories. + +They are not maintained in the same way as the main scripts, and come in many categories including: + +- Scripts under development +- Examples of how to do things +- Deprecated scripts +- Left-overs from previous development plans! + +Note - all FontForge-based scripts need updating, since FontForge (as "FF") is no longer a supported tool for execute() + +Some are documented below. + +## Table of scripts + +| Command | Status | Description | +| ------- | ------ | ----------- | +| [accesslibplist.py](#accesslibplist) | ? | Demo script for accessing fields in lib.plist | +| [chaindemo.py](#chaindemo) | ? | Demo of how to chain calls to multiple scripts together | +| [ffchangeglyphnames](#ffchangeglyphnames) | ? | Update glyph names in a ttf font based on csv file | +| [ffcopyglyphs](#ffcopyglyphs) | ? | Copy glyphs from one font to another, without using ffbuilder | +| [ffremovealloverlaps](#ffremovealloverlaps) | ? | Remove overlap on all glyphs in a ttf font | +| [FFmapGdlNames.py](#ffmapgdlnames) | ? | Write mapping of graphite names to new graphite names | +| [FFmapGdlNames2.py](#ffmapgdlnames2) | ? | Write mapping of graphite names to new graphite names | +| [FLWriteXml.py](#flwritexml) | ? | Outputs attachment point information and notes as XML file for TTFBuilder | +| [FTaddEmptyOT.py](#ftaddemptyot) | ? | Add empty Opentype tables to ttf font | +| [FTMLnorm.py](#ftmlnorm) | ? | Normalize an FTML file | +| [psfaddGlyphDemo.py](#psfaddglyphdemo) | ? | Demo script to add a glyph to a UFO font | +| [psfexpandstroke.py](#psfexpandstroke) | ? | Expands an unclosed UFO stroke font into monoline forms with a fixed width | +| [psfexportnamesunicodesfp.py](#psfexportnamesunicodesfp) | ? | Outputs an unsorted csv file containing the names of all the glyphs in the default layer | +| [psfgenftml.py](#psfgenftml) | ? | generate ftml tests from glyph_data.csv and UFO | +| [psftoneletters.py](#psftoneletters) | ? | Creates Latin script tone letters (pitch contours) | +| [xmlDemo.py](#xmldemo) | ? | Demo script for use of ETWriter | + + +--- +#### accesslibplist +Usage: **` python accesslibplist.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Demo script for accessing fields in lib.plist + + +--- +#### chaindemo +Usage: **` python chaindemo.py ...`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Demo of how to chain calls to multiple scripts together. +Running + +`python chaindemo.py infont outfont --featfile feat.csv --uidsfile uids.csv` + +will run execute() against psfnormalize, psfsetassocfeat and psfsetassocuids passing the font, parameters +and logger objects from one call to the next. So: +- the font is only opened once and written once +- there is a single log file produced + + +--- +#### ffchangeglyphnames +Usage: **`ffchangeglyphnames [-i INPUT] [--reverse] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Update the glyph names in a ttf font based on csv file. + +Example usage: + +``` +ffchangeglyphnames -i glyphmap.csv font.ttf +``` +will update the glyph names in the font based on mapping file glyphmap.csv + +If \-\-reverse is used, it change names in reverse. + +--- +#### ffcopyglyphs +Usage: **`ffcopyglyphs -i INPUT [-r RANGE] [--rangefile RANGEFILE] [-n NAME] [--namefile NAMEFILE] [-a] [-f] [-s SCALE] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +_This section is Work In Progress!_ + +optional arguments: + +``` + -h, --help show this help message and exit + -i INPUT, --input INPUT + Font to get glyphs from + -r RANGE, --range RANGE + StartUnicode..EndUnicode no spaces, e.g. 20..7E + --rangefile RANGEFILE + File with USVs e.g. 20 or a range e.g. 20..7E or both + -n NAME, --name NAME Include glyph named name + --namefile NAMEFILE File with glyph names + -a, --anchors Copy across anchor points + -f, --force Overwrite existing glyphs in the font + -s SCALE, --scale SCALE + Scale glyphs by this factor +``` + +--- +#### ffremovealloverlaps +Usage: **`ffremovealloverlaps ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Remove overlap on all glyphs in a ttf font + +--- +#### FFmapGdlNames +Usage: **` python FFmapGdlNames2.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Write mapping of graphite names to new graphite names based on: + - two ttf files + - the gdl files produced by makeGdl run against those fonts + This could be different versions of makeGdl + - a csv mapping glyph names used in original ttf to those in the new font + + +--- +#### FFmapGdlNames2 +Usage: **` python FFmapGdlNames.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Write mapping of graphite names to new graphite names based on: + - an original ttf font + - the gdl file produced by makeGdl when original font was produced + - a csv mapping glyph names used in original ttf to those in the new font + - pysilfont's gdl library - so assumes pysilfonts makeGdl will be used with new font + + +--- +#### FLWriteXml +Usage: **` python FLWriteXml.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Outputs attachment point information and notes as XML file for TTFBuilder + + +--- +#### FTaddEmptyOT +Usage: **` python FTaddEmptyOT.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Add empty Opentype tables to ttf font + + +--- +#### FTMLnorm +Usage: **` python FTMLnorm.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Normalize an FTML file + + +--- +#### psfaddGlyphDemo +Usage: **` python psfaddGlyphDemo.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Demo script to add a glyph to a UFO font + + +--- +#### psfexpandstroke + +Usage: **`psfexpandstroke infont outfont expansion`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Expands the outlines (typically unclosed) in an UFO stroke font into monoline forms with a fixed width. + +Example that expands the stokes in a UFO font `SevdaStrokeMaster-Regular.ufo` by 13 units on both sides, giving them a total width of 26 units, and writes the result to `Sevda-Regular.ufo`. + +``` +psfexpandstroke SevdaStrokeMaster-Regular.ufo Sevda-Regular.ufo 13 +``` + +Note that this only expands the outlines - it does not remove any resulting overlap. + + +--- +#### psfexportnamesunicodesfp +Usage: **` python psfexportnamesunicodesfp.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Outputs an unsorted csv file containing the names of all the glyphs in the default layer and their primary unicode values. + +Format name,usv + + +--- +#### psfgenftml +Usage: **` python psfgenftml.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +generate ftml tests from glyph_data.csv and UFO + + +--- +#### psftoneletters +Usage: **`psftoneletters infont outfont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This uses the parameters from the UFO lib.plist org.sil.lcg.toneLetters key to create Latin script tone letters (pitch contours). + +Example usage: + +``` +psftoneletters Andika-Regular.ufo Andika-Regular.ufo +``` + + +--- +#### xmlDemo +Usage: **` python xmlDemo.py ...`** + +_([Standard options](docs.md#standard-command-line-options) may also apply)_ + +Demo script for use of ETWriter diff --git a/docs/fea2_proposal.md b/docs/fea2_proposal.md new file mode 100644 index 0000000..aa3ae99 --- /dev/null +++ b/docs/fea2_proposal.md @@ -0,0 +1,150 @@ +# Proposed Extensions to FEA + +This document describes a macro extension to FEA that will enable it to grow +and support more powerful OpenType descriptions. The proposal is presented as +various syntax extensions to the core FEA syntax. + +## Functions + +Currently FEA makes no use of parentheses. This may be a conscious decision to +reserve these for later ues. Such parentheses lend themselves perfectly to the +addition of macro functions to the FEA syntax: + +``` +function = funcname '(' (parameter (',' parameter)*)? ')' + +funcname = /[A-Za-z_.][A-Za-z_0-9.]*/ +parameter = glyph | glyphlist | classref | value_record | function + | ('"' string '"') | ("{" tokens "}") +tokens = noncurlytoken* | ("{" tokens "}") +glyphlist = '[' glyph* ']' +classref = '@' classname +value_record = number | '<' chars '>' +``` + +A function call consists of a function name and a parenthesised parameter list, +which may be empty. + + + and an optional following token list enclosed in braces. The +token list is just that, an unparsed sequence of lexical tokens. The result of +the function is also an unparsed sequence of lexical tokens that are then parsed +and processed as if the function were replaced by a textual representation of +the tokens. + +The parameters are parsed, so for example a classref would expand to its +resulting list of glyphs. Likewise a function call result would be parsed to its +single semantic item, it is not parsed as a token list. A value_record is the +widest interpretation of a value record, including an anchor. Basically it is +either a number or anything between < and >. + +A function statement is the use of a function result as a statement in the FEA +syntax. + +The FEA syntax defines nothing more that functions exist and how they may be +referenced. It is up to a particular FEA processor to supply the functions and +to execute them to resolve them to a token list. It is also up to the particular +FEA processor to report an error or otherwise handle an unknown function +reference. As such this is similar to other programming languages where the +language itself says nothing about what functions exist or what they do. That is +for libraries. + +There is one exception. The `include` statement in the core FEA syntax follows +the same syntax, apart from the missing quotation marks around the filename. As +such `include` is not available for use as a function name. + +### Sample Implementation + +In this section we give a sample implementation based on the FEA library in +fonttools. + +Functions are kept in module style namespaces, much like a simplified python module +system. A function name then typically consists of a `modulename.funcname` The +top level module is reserved for the fea processor itself. The following +functions are defined in the top level module (i.e. no modulename.) + +#### load + +The `load` function takes a path to a file containing python definitions. +Whether this python code is preprocessed for security purposes or not is an open +question. It also takes a modulename as its second parameter. + +``` +load("path/to/pythonfile.py", "mymodule") +``` + +The function returns an empty token string but has the effect of loading all the +functions defined in the python file as those functions prefixed by the +modulename, as described above. + +#### set + +This sets a variable to a token list. Variables are described in a later syntax +extension. The first parameter is the name of a variable. The token list is then +used for the variable expansion. + +``` +set("distance") { 30 }; +``` + +Other non top level module may be supplied with the core FEA processing module. + +#### core.refilter + +This function is passed a glyphlist (or via a classref) and a regular +expression. The result is a glyphlist consisting of all the glyphs whose name +matches the regular expression. For example: + +``` +@csc = core.refilter("\.sc$", @allglyphs) +``` + +#### core.pairup + +This function is passed two classnames, a regular expression and a glyph list. +The result is two class definitions for the two classnames. One class is +of all the glyphs which match the regular expression. The other class is a +corresponding list of glyphs whose name is the same as the matching regular +expression with the matching regular expression text removed. If no such glyph +exists in the font, then neither the name or the glyph matching the regular +expression is included. The resulting classes may therefore be used in a simple +substitution. For example: + +``` +core.pairup("cnosc", "csc", "\.sc$", [a.sc b.sc fred.sc]); +lookup smallcap { + sub @cnosc by @csc; +} smallcap; +``` + +Assuming `fred.sc` exists but `fred` does not, this is equivalent to: + +``` +@cnosc = [a b]; +@csc = [a.sc b.sc]; +lookup smallcap { + sub @cnosc by @csc; +} smallcap; +``` + +## Variables + +A further extension to the FEA syntax is to add a simple variable expansion. A +variable expands to a token list. Since variables may occur anywhere they need a +syntactic identifier. The proposed identifier is an initial `$`. + +``` +variable = '$' funcname +``` + +Variables are expanded at the point of expansion. Since expansion is recursive, +the variable may contain a function call which expands when the variable +expands. + +There is no syntax for defining a variable. This is unnatural and may be +revisited if a suitable syntax can be found. Definition is therefore a processor +specific activity. + +It is undecided whether undefined variables expand to an empty token list or an +error. + diff --git a/docs/feaextensions.md b/docs/feaextensions.md new file mode 100644 index 0000000..053e28e --- /dev/null +++ b/docs/feaextensions.md @@ -0,0 +1,559 @@ +# FEA Extensions Current + +This document describes the functionality of `psfmakefea` and lists the extensions to fea that are currently supported. +<!-- TOC --> + +- [Generated Classes](#generated-classes) + - [Variant glyph classes](#variant-glyph-classes) + - [Ligatures](#ligatures) +- [Statements](#statements) + - [baseclass](#baseclass) + - [Cursive Attachment](#cursive-attachment) + - [Mark Attachment](#mark-attachment) + - [Ligature Attachment](#ligature-attachment) + - [ifinfo](#ifinfo) + - [ifclass](#ifclass) + - [do](#do) + - [SubStatements](#substatements) + - [for](#for) + - [let](#let) + - [forlet](#forlet) + - [if](#if) + - [Examples](#examples) + - [Simple calculation](#simple-calculation) + - [More complex calculation](#more-complex-calculation) + - [Right Guard](#right-guard) + - [Left Guard](#left-guard) + - [Left Kern](#left-kern) + - [Myanmar Great Ya](#myanmar-great-ya) + - [Advance for Ldot on U](#advance-for-ldot-on-u) + - [def](#def) + - [python support](#python-support) + - [kernpairs](#kernpairs) +- [Capabilities](#capabilities) + - [Permit classes on both sides of GSUB type 2 (multiple) and type 4 (ligature) lookups](#permit-classes-on-both-sides-of-gsub-type-2-multiple-and-type-4-ligature-lookups) + - [Processing](#processing) + - [Example](#example) + - [Support classes in alternate lookups](#support-classes-in-alternate-lookups) + - [groups.plist](#groupsplist) + +<!-- /TOC --> +## Generated Classes + +`psfmakefea` simplifies the hand creation of fea code by analysing the glyphs in the input font, particularly with regard to their names. Names are assumed to conform to the Adobe Glyph List conventions regarding `_` for ligatures and `.` for glyph variants. + +### Variant glyph classes + +If a font contains a glyph with a final variant (there may be more than one listed for a glyph, in sequence) and also a glyph without that final variant, then `psfmakefea` will create two classes based on the variant name: @c\__variant_ contains the glyph with the variant and @cno\__variant_ contains the glyph without the variant. The two lists are aligned such that a simple classes based replacement will change all the glyphs without the variant into ones with the variant. + +For example, U+025B is an open e that occurs in some African languages. Consider a font that contains the glyphs `uni025B` and `uni025B.smcp` for a small caps version of the glyph. `psfmakefea` will create two classes: + +``` +@c_smcp = [uni025B.scmp]; +@cno_smcp = [uni025B]; +``` + +In addition, if this font contains two other glyphs `uni025B.alt`, an alternative shape to `uni025B` and `uni025B.alt.smcp`, the small caps version of the alternate. `psfmakefea` will create the following classes: + +``` +@c_smcp = [uni025B.scmp uni025B.alt.smcp]; +@cno_smcp = [uni025B uni025B.alt]; +@c_alt = [uni025B.alt]; +@cno_alt = [uni025B]; +``` + +Notice that the classes with multiple glyphs, while keeping the alignment, do not guarantee any particular order of the glyphs in one of the classes. Only that the other class will align its glyph order correctly. Notice also that `uni025B.alt.smcp` does not appear in the `@c_alt` class. This latter behaviour may change. + +### Ligatures + +Unless instructed on the command line via the `-L` or `--ligmode` option, `psfmakefea` does nothing special with ligatures and treats them simply as glyphs that may take variants. There are four ligature modes. The most commonly used is `-L last`. This says to create classes based on the last components in all ligatures. Thus if the font from the previous section also included `uni025B_acutecomb` and the corresponding small caps `uni025B_acutecomb.smcp`. We also need an `acutecomb`. If the command line included `-L last`, the generated classes would be: + +``` +@c_smcp = [uni025B.scmp uni025B.alt.smcp uni025B_acutecomb.smcp]; +@cno_smcp = [uni025B uni025B.alt uni025B_acutecomb]; +@c_alt = [uni025B.alt]; +@cno_alt = [uni025B]; +@clig_acutecomb = [uni025B_acutecomb]; +@cligno_acutecomb = [uni025B]; +``` + +And if the command line option were `-L first`, the last two lines of the above code fragment would become: + +``` +@clig_uni025B = [uni025B_acutecomb]; +@cligno_uni025B = [acutecomb]; +``` + +while the variant classes would remain the same. + +There are two other ligaturemodes: `lastcomp` and `firstcomp`. These act like `last` and `first`, but in addition they say that any final variants must be handled differently. Instead of seeing the final variants (those on the last ligature component) as applying to the whole ligature, they are only to be treated as applying to the last component. To demonstrate this we need to add the nonsensical `acutecomb.smcp`. With either `-L last` or `-L first` we get the same ligature classes as above. (Although we would add `acutecomb.smcp` to the `@c_smcp` and `acutecomb` to `@cno_smcp`) With `-L firstcomp` we get: + +``` +@c_smcp = [uni025B.scmp uni025B.alt.smcp acutecomb.smcp]; +@cno_smcp = [uni025B uni025B.alt acutecomb]; +@c_alt = [uni025B.alt]; +@cno_alt = [uni025B]; +@clig_uni025B = [uni025B_acutecomb uni025B_acutecomb.smcp]; +@cligno_uni025B = [acutecomb acutecomb.smcp]; +``` + +Notice the removal of `uni025B_acutecomb.smcp` from `@c_smcp`, since `uni025B_acutecomb.smcp` is considered by `-L firstcomp` to be a ligature of `uni025B` and `acutecomb.smcp` there is no overall ligature `uni025B_acutecomb` with a variant `.smcp` that would fit into `@c_smcp`. If we use `-L lastcomp` we change the last two classes to: + +``` +@clig_acutecomb = [uni025B_acutecomb]; +@cligno_acutecomb = [uni025B]; +@clig_acutecomb_smcp = [uni025B_acutecomb.smcp]; +@cligno_acutecomb_smcp = [un025B]; +``` + +With any `.` in the variant being changed to `_` in the class name. + +In our example, if the author wanted to use `-L lastcomp` or `-L firstcomp`, they might find it more helpful to rename `uni025B_acutecomb.smcp` to `uni025B.smcp_acutecomb` and remove the nonsensical `acutecomb.smcp`. This would give, for `-L lastcomp`: + +``` +@c_smcp = [uni025B.scmp uni025B.alt.smcp]; +@cno_smcp = [uni025B uni025B.alt]; +@c_alt = [uni025B.alt]; +@cno_alt = [uni025B]; +@clig_acutecomb = [uni025B_acutecomb uni025B.smcp_acutecomb]; +@cligno_acutecomb = [uni025B uni025B.smcp]; +``` + +and for `-L firstcomp`, the last two classes become: + +``` +@clig_uni025B = [uni025B_acutecomb]; +@cligno_uni025B = [acutecomb]; +@clig_uni025B_smcp = [uni025B.smcp_acutecomb]; +@cligno_uni025B_smcp = [acutecomb]; +``` + +## Statements + +### baseclass + +A baseclass is the base equivalent of a markclass. It specifies the position of a particular class of anchor points on a base, be that a true base or a mark base. The syntax is the same as for a markclass, but it is used differently in a pos rule: + +``` +markClass [acute] <anchor 350 0> @TOP_MARKS; +baseClass [a] <anchor 500 500> @BASE_TOPS; +baseClass b <anchor 500 750> @BASE_TOPS; + +feature test { + pos base @BASE_TOPS mark @TOP_MARKS; +} test; +``` + +Which is the functional equivalent of: + +``` +markClass [acute] <anchor 350 0> @TOP_MARKS; + +feature test { + pos base [a] <anchor 500 500> mark @TOP_MARKS; + pos base b <anchor 500 750> mark @TOP_MARKS; +} test; +``` + +It should be borne in mind that both markClasses and baseClasses can also be used as normal glyph classes and as such use the same namespace. + +The baseClass statement is a high priority need in order to facilitate auto generation of attachment point information without having to create what might be redundant lookups in the wrong order. + +Given a set of base glyphs with attachment point A and marks with attachment point \_A, psfmakefea will generate the following: + +- baseClass A - containing all bases with attachment point A +- markClass \_A - containing all marks with attachment point \_A +- baseClass A\_MarkBase - containing all marks with attachment point A + +#### Cursive Attachment + +Cursive attachment involves two base anchors, one for the entry and one for the exit. We can extend the use of baseClasses to support this, by passing two baseClasses to the pos cursive statement: + +``` +baseClass meem.medial <anchor 700 50> @ENTRIES; +baseClass meem.medial <anchor 0 10> @EXITS; + +feature test { + pos cursive @ENTRIES @EXITS; +} test; +``` + +Here we have two base classes for the two anchor points, and the pos cursive processing code works out which glyphs are in both classes, and which are in one or the other and generates the necessary pos cursive statement for each glyph. I.e. there will be statements for the union of the two classes but with null anchors for those only in one (according to which baseClass they are in). This has the added advantage that any code generating baseClasses does not need to know whether a particular attachment point is being used in a cursive attachment. That is entirely up to the user of the baseClass. + +#### Mark Attachment + +The current mark attachment syntax is related to the base mark attachment in that the base mark has to be specified explicitly and we cannot currently use a markclass as the base mark in a mark attachment lookup. We can extend the mark attachment in the same way as we extend the base attachment, by allowing the mark base to be a markclass. Thus: + +``` +pos mark @MARK_BASE_CLASS mark @MARK_MARK_CLASS; +``` + +Would expand out to a list of mark mark attachment rules. + +#### Ligature Attachment + +Ligature attachment involves all the attachments to a ligature in a single rule. Given a list of possible ligature glyphs, the ligature positioning rule has been extended to allow the use of baseClasses instead of the base anchor on the ligature. For a noddy example: + +``` +baseClass a <anchor 200 200> @TOP_1; +baseClass fi <anchor 200 0> @BOTTOM_1; +baseClass fi <anchor 400 0> @BOTTOM_2; +markClass acute <anchor 0 200> @TOP; +markClass circumflex <anchor 200 0> @BOTTOM; + +pos ligature [a fi] @BOTTOM_1 mark @BOTTOM @TOP_1 mark @TOP + ligComponent @BOTTOM_2 mark @BOTTOM; +``` + +becomes + +``` +pos ligature a <anchor 200 200> mark @TOP + ligComponent <anchor NULL>; +pos ligature fi <anchor 200 0> mark @BOTTOM + ligComponent <anchor 400 0> mark @BOTTOM; +``` + +### ifinfo + +This statement initiates a block either of statements or within another block. The block is only processed if the ifinfo condition is met. ifinfo takes two parameters. The first is a name that is an entry in a fontinfo.plist. The second is a string containing a regular expression that is matched against the given value in the fontinfo.plist. If there is a match, the condition is considered to be met. + +``` +ifinfo(familyName, "Doulos") { + +# statements + +} +``` + +Notice the lack of a `;` after the block close. + +ifinfo acts as a kind of macro, this means that the test is executed in the parser rather than collecting everything inside the block and processing it later like say the `do` statement. Notice that if you want to do something more complex than a regular expression test, then you may need to use a `do` statement and the `info()` function. + +### ifclass + +This statement initiates a block either of statements or within another block. The block is only processed if the given @class is defined and contains at least one glyph. + +``` +ifclass(@oddities) { + +# statements + +} +``` + +Notice the lack of a `;` after the block close. + +### do + +The `do` statement is a means of setting variables and repeating statement groups with variable expansion. A `do` statement is followed by various substatements that are in effect nested statements. The basic structure of the `do` statement is: + +`do` _substatement_ _substatement_ _..._ [ `{` _statements_ `}` ] + +Where _statements_ is a sequence of FEA statements. Within these statements, variables may be referenced by preceding them with a `$`. Anything, including statement words, can be the result of variable expantion. The only constraints are: + +- The item expands to one or more complete tokens. It cannot be joined to something preceding or following it to create a single name, token, whatever. + +In effect a `{}` type block following a `for` or `let` substatement is the equivalent of inserting the substatement `if True;` before the block. + +#### SubStatements + +Each substatement is terminated by a `;`. The various substatements are: + +##### for + +The `for` substatement is structured as: + +`for` _var_ `=` _glyphlist_ `;` + +This creates a variable _var_ that will iterate over the _glyphlist_. + +With the addition of `forlet` (see below), there is also `forgroup` that is a synonym for the `for` substatement defined here. + +##### let + +The `let` substatement executes a short python expression (via `eval`), storing the result in the given variable, or variable list. The structure of the substatement is: + +`let` _var_ [`,` _var_]* `=` _expression_ `;` + +There are various python functions that are especially supported, along with the builtins. These are: + +| Function | Parameters | Description | +|-------------|-------------|------------------------| +| ADVx | _glyphname_ | Returns the advanced width of the given glyph | +| allglyphs | | Returns a list of all the glyph names in the font | +| APx | _glyphname_, "_apname_" | Returns the x coordinate of the given attachment point on the given glyph | +| APy | _glyphname_, "_apname_" | Returns the y coordinate of the given attachment point on the given glyph | +| feaclass | _classname_ | Returns a list of the glyph names in a class as a python list | +| info | _finfoelement_ | Looks up the entry in the fontinfo plist and returns its value | +| kerninfo | | Returns a list of tuples (left, right, kern_value) | +| opt | _defined_ | Looks up a given -D/--define variable. Returns empty string if missing | +| MINx | _glyphname_ | Returns the minimum x value of the bounding box of the glyph | +| MINy | _glyphname_ | Returns the minimum y value of the bounding box of the glyph | +| MAXx | _glyphname_ | Returns the maximum x value of the bounding box of the glyph | +| MAXy | _glyphname_ | Returns the maximum y value of the bounding box of the glyph | + +See the section on python in the `def` command section following. + +##### forlet + +The `for` substatement only allows iteration over a group of glyphs. There are situations in which someone would want to iterate over a true python expression, for example, over the return value of a function. The `forlet` substatement is structured identically to a `let` substatement, but instead of setting the variable once, the following substatements are executed once for each value of the expression, with the variable set to each in turn. For example: + +``` +def optlist(*alist) { + if len(alist) > 0: + for r in optlist(*alist[1:]): + yield [alist[0]] + r + yield r + else: + yield alist +} optlist; + +lookup example { +do + forlet l = optlist("uni17CC", "@coeng_no_ro", "[uni17C9 uni17CA]", "@below_vowels", "@above_vowels"); + let s = " ".join(l) + { + sub uni17C1 @coeng_ro @base_cons $s uni17B8' lookup insert_dotted_circle; + sub uni17C1 @base_cons $s uni17B8' lookup insert_dotted_circle; + } +} example; +``` + +This examples uses a `def` statement as defined below. The example produces rules for each of the possible subsequences of the optlist parameters, where each element is treated as being optional. It is a way of writing: + +``` +sub uni17C1 @base_cons uni17CC? @coeng_no_ro [uni17C9 uni17CA]? @below_vowels? @above_vowels? uni17B8' lookup insert_dotted_circle; +``` + +The structure of a `forlet` substatement is: + +`forlet` _var_ [`,` _var_]* `=` _expression_ `;` + +A `forlet` substatement has the same access to functions that the `let` statement has, included those listed above under `let`. + + +##### if + +The `if` substatement consists of an expression and a block of statements. `if` substatements only make sense at the end of a sequence of substatements and are executed at the end of the `do` statement, in the order they occur but after all other `for` and `let` substatements. The expression is calculated and if the result is True then the _statements_ are expanded using variable expansion. + +`if` _expression_ `;` `{` _statements_ `}` + +There can be multiple `if` substatements, each with their own block, in a `do` statement. + +#### Examples + +The `do` statement is best understood through some examples. + +##### Simple calculation + +This calculates a simple offset shift and creates a lookup to apply it: + +``` +do let a = -int(ADVx("u16F61") / 2); + { + lookup left_shift_vowel { + pos @_H <$a 0 0 0>; + } left_shift_vowel; + } +``` + +Notice the lack of iteration here. + +##### More complex calculation + +This calculates the guard spaces on either side of a base glyph in response to applied diacritics. + +``` +lookup advance_base { +do for g = @H; + let a = APx(g, "H") - ADVx(g) + int(1.5 * ADVx("u16F61")); + let b = int(1.5 * ADVx("u16F61")) - APx(g, "H"); + let c = a + b; + { + pos $g <$b 0 $c 0>; + } +} advance_base; +``` + +##### Right Guard + +It is often desirable to give a base character extra advance width to account for a diacritic hanging over the right hand side of the glyph. Calculating this can be very difficult by hand. This code achieves this: + +``` +do for b = @bases; + for d = @diacritics; + let v = (ADVx(d) - APx(d, "_U")) - (ADVx(b) - APx(b, "U")); + if v > 0; { + pos $b' $v $d; + } +``` + +##### Left Guard + +A corresponding guarding of space for diacritics may be done on the left side of a glyph: + +``` +do for b = @bases; + for d = @diacritics; + let v = APx(d, "_U") - APx(b, "U"); + if v > 0; { + pos $b' <$v 0 $v 0> $d; + } +``` + +##### Left Kern + +Consider the case where someone has used an attachment point as a kerning point. In some context they want to adjust the advance of the left glyph based on the position of the attachment point in the right glyph: + +``` +do for r = @rights; + let v = APx(r, "K"); { + pos @lefts' $v $r; + pos @lefts' $v @diacritics $r; + } +``` + +##### Myanmar Great Ya + +One obscure situation is the Great Ya (U+103C) in the Myanmar script, that visual wraps around the following base glyph. The great ya is given a small advance to then position the following consonant glyph within it. The advance of this consonant needs to be enough to place the next character outside the great ya. So we create an A attachment point on the great ya to emulate this intended final advance. Note that there are many variants of the great ya glyph. Thus: + +``` +do for y = @c103C_nar; + for c = @cCons_nar; + let v = APx(y, "A") - (ADVx(y) + ADVx(c)); + if v > 0; { + pos $y' $v $c; + } + +do for y = @c103C_wide; + for c = @cCons_wide; + let v = APx(y, "A") - (ADVx(y) + ADVx(c)); + if v > 0; { + pos $y' $v $c; + } +``` + +##### Advance for Ldot on U + +This example mirrors that used in the proposed [`setadvance`](feax_future.md#setadvance) statement. Here we want to add sufficient advance on the base to correspond to attaching an u vowel which in turn has a lower dot attached to it. + +``` +do for b = @cBases; + for u = @cLVowels; + let v = APx(b, "L") - APx(u, "_L") + APx(u, "LD") - APx("ldot", "_LD") + ADVx("ldot") - ADVx(b); + if v > 0; { + pos $b' $v $u ldot; + } +``` + +### def + +The `def` statement allows for the creation of python functions for use in `let` substatements of the `do` statement. The syntax of the `def` statement is: + +``` +def <fn>(<param_list>) { + ... python code ... +} <fn>; +``` + +The `fn` must conform to a FEA name (not starting with a digit, etc.) and is repeated at the end of the block to mark the end of the function. The parameter is a standard python parameter list and the python code is standard python code, indented as if under a `def` statement. + +#### python support +Here and in `let` and `forlet` substatements, the python that is allowed to be executed is limited. Only a subset of functions from builtins is supported and the `__` may not occur in any attribute. This is to stop people escaping the sandbox in which python code is interpreted. The `math` and `re` modules are also included along with the functions available to a `let` and `forlet` substatement. The full list of builtins supported are: + +``` +True, False, None, int, float, str, abs, bool, dict, enumerate, filter, hex, isinstance, len, list, +map, max, min, ord, range, set, sorted, sum, tuple, type, zip +``` + +### kernpairs + +The `kernpairs` statement expands all the kerning pairs in the font into `pos` statements. For example: + +``` +lookup kernpairs { + lookupflag IgnoreMarks; + kernpairs; +} kernpairs; +``` + +Might produce: + +``` +lookup kernpairs { + lookupflag IgnoreMarks; + pos @MMK_L_afii57929 -164 @MMK_R_uniA4F8; + pos @MMK_L_uniA4D1 -164 @MMK_R_uniA4F8; + pos @MMK_L_uniA4D5 -164 @MMK_R_afii57929; + pos @MMK_L_uniA4FA -148 @MMK_R_space; +} kernpairs; +``` + +Currently, kerning information is only available from .ufo files. + +## Capabilities + +### Permit classes on both sides of GSUB type 2 (multiple) and type 4 (ligature) lookups + +Adobe doesn't permit compact notation using groups in 1-to-many (decomposition) rules e.g: + +``` + sub @AlefPlusMark by absAlef @AlefMark ; +``` + +or many-to-1 (ligature) rules, e.g.: + +``` + sub @ShaddaKasraMarks absShadda by @ShaddaKasraLigatures ; +``` + +This is implemented in FEAX as follows. + +#### Processing + +Of the four simple (i.e., non-contextual) substitution lookups, Types 2 and 4 +are the only ones using the 'by' keyword that have a *sequence* of glyphs or +classes on one side of the rule. The other side will, necessarily, contain a +single term -- which Adobe currently requires to be a glyph. For convenience of +expression, we'll call the sides of the rule the *sequence side* and the *singleton side*. + +* Non-contextual substitution +* Uses the 'by' keyword +* Singleton side references a glyph class. + +Such rules are expanded by enumerating the singleton side class and the corresponding +class(es) on the sequence side and writing a set of Adobe-compliant rules to give +the same result. It is an error if the singleton and corresponding classes do +not have the same number of glyphs. + +#### Example + +Given: + +``` + @class1 = [ g1 g2 ] ; + @class2 = [ g1a g2a ] ; +``` + +then + +``` + sub @class1 gOther by @class2 ; +``` + +would be rewritten as: + +``` + sub g1 gOther by g1a ; + sub g2 gOther by g2a ; +``` + +### Support classes in alternate lookups + +The default behaviour in FEA is for a `sub x from [x.a x.b];` to only allow a single glyph before the `from` keyword. But it is often useful to do things like: `sub @a from [@a.lower @a.upper];`. Feax supports this by treating the right hand side list of glyphs as a single list and dividing it equally by the list on the left. Thus if `@a` is of length 3 then the first 3 glyphs in the right hand list will go one each as the first alternate for each glyph in `@a`, then the next 3 go as the second alternate, and so on until they are all consumed. If any are left over in that one of the glyphs ends up with a different number of alternates to another, then an error is given. + +### groups.plist + +If a .ufo file contains a `groups.plist` file, the groups declared there are propagated straight through to the output file and can be referenced within a source file. + diff --git a/docs/feax_future.md b/docs/feax_future.md new file mode 100644 index 0000000..a989a5d --- /dev/null +++ b/docs/feax_future.md @@ -0,0 +1,283 @@ +# FEA Extensions Future + +## Introduction + +This document is where people can dream of the extensions they would like to see +added to FEA. Notice that any extensions need to be convertible back to normal FEA +so shouldn't do things that can't be expressed in FEA. + +As things get implemented from here, they will be moved to feaextensions.md. There +are no guaranteees that what is in here, will end up in psfmakefea. +The various features listed here are given priorities: + +| Level | Priority +|------|------------- +| 1 | Intended to be implemented +| 2 | Probably will be implemented but after priority 1 stuff +| 3 | Almost certainly won't be implemented + +There are a number of possible things that can be added to FEA, the question is whether they are worth adding in terms of meeting actual need (remove from this list if added to the rest of the document): + +* classsubtract() classand() functions + * classand(x, y) = classsubtract(x, (classsubtract(x, y)) +* classbuild(class, "$.ext") builds one class out of another. What if something is missing? Or do we just build those classes on the fly from make_fea and glyph name parsing? + +## Statements + +Statements are used to make rules, lookups, etc. + +### setadvance + +Priority: 3 (since the do statement has a higher priority and covers this) + +This function does the calculations necessary to adjust the advance of a glyph based on information of attachment points, etc. The result is a single shift on each of the glyphs in the class. The syntax is: + +``` +setadvance(@glyphs, APName [, attachedGlyph[, APName, attachedGlyph [...]]]) +``` + +In effect there are two modes for this function. The first only has two parameters +and shifts the advance from its default designed position to the x coordinate of +the given attachment point. The second mode adds extra glyphs. The advance is moved +to the advance of the attachedGlyph assuming the base has the other glyphs chained +attached at their given APs. An AP may be a number in which case that is the +x coordinate of the AP that will be used. + +Typically there will be only one of these per lookup, unless the classes referenced +are non overlapping. + +The statement only triggers if the resulting advance is greater than the current +advance. Thus some glyphs may not have a statement created for them. I.e. all +values in the lookup will be positive. + +#### Examples + +These examples also act as motivating use cases. + +##### Nokyung + +In Nokyung there is a need to kern characters that do not descend below the baseline closer to glyphs with a right underhang. This can be done through kerning pairs or we could add an attachment point to the glyphs with the right underhang and contextual adjust their advances to that position. The approach of using an AP to do kerning is certainly quirky and few designers would go that route. The contextual lookup would call a lookup that just does the single adjustment. Consider the AP to be called K (for kern). The fea might look like: + +``` +lookup overhangKernShift { + setadvance(@overhangs, K); +} overhangKernShift; +``` + +And would expand, potentially, into + +``` +lookup overhangKernShift { + @overhangs <-80>; +} overhangKernShift; +``` +Not much, but that is because in Nokyung the overhanging glyphs all have the same overhang. If they didn't, then the list could well expand with different values for each glyph in the overhangs class. In fact, a simple implementation would do such an expansion anyway, while a more sophisticated implementation would group the results into ad hoc glyph lists. + +##### Myanmar + +An example from Myanmar is where a diacritic is attached such that the diacritic overhangs the right hand side of the base glyph and we want to extend the advance of the base glyph to encompass the diacritic. This is a primary motivating example for this statement. Such a lookup might read: + +``` +lookup advanceForLDotOnU { + setadvance(@base, L, uvowel, LD, ldot); +} advanceForLDotOnU; +``` + +Which transforms to: + +``` +lookup advanceForLDotOnU { + ka <120>; + kha <80>; +# … +} advanceForLDotOnU; +``` + +##### Miao + +Miao is slightly different in that the advance we want to use is a constant, +partly because calculating it involves a sequence of 3 vowel widths and you end up +with a very long list of possible values and lookups for each one: + +``` +lookup advancesShortShortShort { + setadvance(@base, 1037); +} advancesShortShortShort; +``` + +#### Issues + +* Do we want to use a syntax more akin to that used for composites, since that is, in effect, what we are describing: make the base have the advance of the composite? +* Do we want to change the output to reflect the sequence so that there can be more statements per lookup? + * The problem is that then you may want to skip intervening non-contributing glyphs (like upper diacritics in the above examples), which you would do anyway from the contextual driving lookup, but wouldn't want to have to do in each situation here. +* It's a bit of a pain that in effect there is only one setadvance() per lookup. It would be nice to do more. +* Does this work (and have useful meaning) in RTL? +* Appears to leave the base glyph *position* unchanged. Is there a need to handle, for example in LTR scripts, LSB change for a base due to its diacritics? (Think i-tilde, etc.) + +### move + +Priority: 2 + +The move semantic results in a complex of lookups. See this [article](https://github.com/OpenType/opentype-layout/blob/master/docs/ligatures.md) on how to implement a move semantic successfully in OpenType. As such a move semantic can only be expressed as a statement at the highest level since it creates lookups. The move statement takes a number of parameters: + +``` +move lookup_basename, skipped, matched; +``` + +The *lookup_basename* is a name (unadorned string) prefix that is used in the naming of the lookups that the move statement creates. It also allows multiple move statements to share the same lookups where appropriate. Such lookups can be referenced by contextual chaining lookups. The lookups generated are: + +| | | +| ---------------------------- | -------------------------------------------------- | +| lookup_basename_match | Contextual chaining lookup to drive the sublookups | +| lookup_basename_pres_matched | Converts skipped(1) to matched + skipped(1) | +| lookup_basename_pref_matched | Converts skipped(1) to matched + skipped(1) + matched | +| lookup_basename_back | Converts skipped(-1) + matched to skipped(-1). | + +Multiple instances of a move statement that use the same *lookup_basename* will correctly merge the various rules in the the lookups created since often at least parts of the *skipped* or *matched* will be the same across different statements. + +Since lookups may be added to, extra contextual rules can be added to the *lookup_basename*_match. + +*skipped* contains a sequence of glyphs (of minimum length 1), where each glyph may be a class or whatever. The move statement considers both the first and last glyph of this sequence when it comes to the other lookups it creates. *skipped(1)* is the first glyph in the sequence and *skipped(-1)* is the last. + +*matched* is a single glyph that is to be moved. There needs to be a two lookups for each matched glyph. + +Notice that only *lookup_basename*_matched should be added to a feature. The rest are sublookups and can be in any order. The *lookup_basename*_matched lookup is created at the point of the first move statement that has a first parameter of *lookup_basename*. + +#### Examples + +While there are no known use cases for this in our fonts at the moment, this is an important statement in terms of showing how complex concepts of wider interest can be implemented as extensions to fea. + +##### Myanmar + +Moving prevowels to the front of a syllable from their specified position in the sequence, in a DFLT processor is one such use of a move semantic: + +``` +move(pv, @cons, my-e); +move(pv, @cons @medial, my-e); +move(pv, @cons @medial @medial, my-e); +move(pv, @cons @medial @medial @medial, my-e); +move(pv, @cons, my-shane); +move(pv, @cons, @medial, my-shane); +``` + +This becomes: + +``` +lookup pv_pres_my-e { + sub @cons by my-e @cons; +} pv_pres_my-e; + +lookup pv_pref_my-e { + sub @cons by my-e @cons my-e; +} pv_pref_my-e; + +lookup pv_back { + sub @cons my-e by @cons; + sub @medial my-e by @medial; + sub @cons my-shane by @cons; + sub @medial my-shane by @medial; +} pv_back; + +lookup pv_match { + sub @cons' lookup pv_pres-my-e my-e' lookup pv_back; + sub @cons' lookup pv_pref-my-e @medial my-e' lookup pv_back; + sub @cons' lookup pv_pref-my-e @medial @medial my-e' lookup pv_back; + sub @cons' lookup pv_pref-my-e @medial @medial @medial my-e' lookup pv_back; + sub @cons' lookup pv_pres-my-shane my-shane' lookup pv_back; + sub @cons' lookup pv_pref-my-shane @medial my-shane' lookup pv_back; +} pv_match; + +lookup pv_pres_my-shane { + sub @cons by my-shane @cons; +} pv_pres_my-shane; + +lookup pv_pref_my-shane { + sub @cons by my-shane @cons my-shane; +} pv_pref_my-shane; +``` + +##### Khmer Split Vowels + +Khmer has a system of split vowels, of which we will consider a very few: + +``` +lookup presplit { + sub km-oe by km-e km-ii; + sub km-ya by km-e km-yy km-ya.sub; + sub km-oo by km-e km-aa; +} presplit; + +move(split, @cons, km-e); +move(split, @cons @medial, km-e); +``` + +## Functions + +Functions may be used in the place of a glyph or glyph class and return a list of glyphs. + +### index + +Priority: 2 + +Used in rules where the expansion of a rule results in a particular glyph from a class being used. Where two classes need to be synchronised, and which two classes are involved, this function specifies the rule element that drives the choice of glyph from this class. This function is motivated by the Keyman language. The parameters of index() are: + +``` +index(slot_index, glyphclass) +``` + +*slot_index* considers the rule as two sequences of slots, each slot referring to one glyph or glyphclass. The first sequence is on the left hand side of the rule and the second on the right, with the index running sequentially from one sequence to the other. Thus if a rule has 2 slots on the left hand side and 3 on the right, a *slot_index* of 5 refers to the last glyph on the right hand side. *Slot_index* values start from 1 for the first glyph on the left hand side. + +What makes an index() function difficult to implement is that it requires knowledge of its context in the statement it occurs in. This is tricky to implement since it is a kind of layer violation. It doesn't matter how an index() type function is represented syntactically, the same problem applies. + +### infont + +Priority: 2 + +This function filters the glyph class that is passed to it, and returns only those glyphs, in glyphclass order, which are actually present in the font being compiled for. For example: + +``` +@cons = infont([ka kha gha nga]); +``` + +## Capabilities + +### Permit multiple classes on RHS of GSUB type 2 (multiple) and the LHS of type 4 (ligature) lookups + +Priority: 2 + +#### Slot correspondence + +In Type 2 (multiple) substitutions, the LHS will be the singleton case and the RHS will be the sequence. In normal use-cases exactly one slot in the RHS will be a class -- all the others will be glyphs -- in which case that class and the singleton side class correspond. + +If more than one RHS slot is to contain a class, then the only logical meaning is that all such classes must also correspond to the singleton class in the LHS, and will be expanded (along with the singleton side class) in parallel. Thus all the classes must have the same number of elements. + +In Type 4 (ligature) substitutions, the RHS will be the singleton class. In the case that the LHS (sequence side) of the rule has class references in more than one slot, we need to identify which slot corresponds to the singleton side class. Some alternatives: + +* Pick the slot that, when the classes are flattened, has the same number of glyphs as the class on the singleton side. It is possible that there is more than one such slot, however. +* Add a notation to the rule. Graphite uses the $n modifier on the RHS to identify the corresponding slot (in the context), which we could adapt to FEA as: + +``` + sub @class1 @class2 @class3 by @class4$2 ; +``` + +Alternatively, since there can be only one such slot, we could use a simpler notation by putting something like the $ in the LHS: + +``` + sub @class1 @class2$ @class3 by @class4 ; +``` + +[This won't look right to GDL programmers, but makes does sense for OT code] + +* Extra syntactic elements at the lexical level are hard to introduce. Instead a function such as: + +``` +sub @class1 @class2 @class3 by index(2, @class4); +``` + +Would give the necessary interpretation. See the discussion of the index() function for more details. + +Note that the other classes in the LHS of ligature rules do not need further processing since FEA allows such classes. + +#### Nested classes + +We will want to expand nested classes in a way (i.e., depth or breadth first) that is compatible with Adobe. **Concern:** Might this be different than Graphite? Is there any difference if one says always expand left to right? [a b [c [d e] f] g] flattens the same as [[[a b] c d] e f g] or whatever. The FontTools parser does not support nested glyph classes. To what extent are they required? diff --git a/docs/parameters.md b/docs/parameters.md new file mode 100644 index 0000000..cec8988 --- /dev/null +++ b/docs/parameters.md @@ -0,0 +1,179 @@ +# Pysilfont parameters + +In addition to normal command-line arguments (see [scripts.md](scripts.md) and [Standard Command-line Options](docs.md#standard-command-line-options)), Pysilfont supports many other parameters that can be changed either on the command-line or by settings in a config file. For UFO fonts there is also an option to set parameters within the UFO. + +See [List of Parameters](#list-of-parameters) for a full list, which includes the default values for each parameter. + +# Setting parameters + +Parameters can be set in multiple ways +1. Default values are set by the core.py Pysilfont module - see [List of Parameters](#list-of-parameters) +1. Standard values for a project can be set in a pysilfont.cfg [config file](#config-file) +1. For UFO fonts, font-specific values can be set within the [lib.plist](#lib-plist) file +1. On the command line \- see next section + +Values set by later methods override those set by earlier methods. + +(Scripts can also change some values, but they would normally be written to avoid overwriting command-line values) + +## Command line + +For script users, parameters can be set on the command line with -p, for example: +``` +psfnormalize test.ufo -p scrlevel=V -p indentIncr=" " +``` +would increase the screen reporting level to Verbose and change the xml indent from 2 spaces to 4 spaces. + +If a parameter has multiple values, enter them separated with commas but no spaces, eg: + +`-p glifElemOrder=unicode,advance,note,image,guideline,anchor,outline,lib` + + + +## Config file +If pysilfont.cfg exists in the same directory as the first file specified on the command line (typically the font being processed) then parameters will be read from there. + +The format is a [ConfigParser](https://docs.python.org/2/library/configparser.html) config file, which is similar structure to a Windows .ini file. + +Lines starting with # are ignored, as are any blank lines. + +Example: +``` +# Config file + +[logging] +scrlevel: I + +[outparams] +indentIncr: ' ' +glifElemOrder: unicode,advance,note,image,guideline,anchor,outline,lib +``` +The section headers are backups, logging, outparams and ufometadata. + +In a font project with multiple UFO fonts in the same folder, all would use a single config file. + +## lib plist + +If, with a UFO font, org.sil.pysilfontparams exists in lib.plist, parameter values held in an array will be processed, eg +``` +<key>org.sil.pysilfontparams</key> +<array> + <indentIncr>\t</indentIncr> + <glifElemOrder>lib,unicode,note,image,guideline,anchor,outline,advance</glifElemOrder> +</array> +``` +Currently only font output parameters can be changed via lib.plist + +## List of parameters + +| Parameter | Default | Description | Notes | +| -------- | -------- | --------------------------------------------- | ------------------------------------- | +| **Reporting** | | | To change within a script use <br>`logger.<parameter> = <value>`| +| scrlevel | P | Reporting level to screen. See [Reporting](docs.md#reporting) for more details | -q, --quiet option sets this to S | +| loglevel | W | Reporting level to log file | | +| **Backup** (font scripts only) | | | | +| backup | True | Backup font to subdirectory | If the original font is being updated, make a backup first | +| backupdir | backups | Sub-directory name for backups | | +| backupkeep | 5 | Number of backups to keep | | +| **Output** (UFO scripts only) | | | To change in a script use <br>`font.outparams[<parameter>] = <value>` | +| indentFirst | 2 spaces | Increment for first level in xml | | +| indentIncr | 2 spaces | Amount to increment xml indents | | +| indentML | False | Indent multi-line text items | (indenting really messes some things up!) | +| plistIndentFirst | Empty string | Different initial indent for plists | (dict is commonly not indented) | +| sortDicts | True | sort all plist dicts | | +| precision | 6 | decimal precision | | +| renameGlifs | True | Name glifs with standard algorithm | | +| UFOversion | (existing) | | Defaults to the version of the UFO when opened | +| format1Glifs | False| Force output of format 1 glifs | Includes UFO2-style anchors; for use with FontForge | +| floatAttribs | (list of attributes in the spec that hold numbers and are handled as float) | Used to know if precision needs setting. | May need items adding for lib data | +| intAttribs | (list of attributes in the spec that hold numbers and handled as integer) | | May need items adding for lib data | +| glifElemOrder | (list of elements in the order defined in spec) | Order for outputting elements in a glif | | +| attribOrders | (list of attribute orders defined in spec) | Order for outputting attributes in an element. One list per element type | When setting this, the parameter name is `attribOrders.<element type>`. Currently only used with attribOrders.glif | +| **ufometadata** (ufo scripts only) | | | | +| checkfix | check | Metadata check & fix action | If set to "fix", some values updated (or deleted). Set to "none" for no metadata checking | +| More may be added... | | + +## Within basic scripts +### Accessing values +If you need to access values of parameters or to see what values have been set on the command line you can look at: +- args.paramsobj.sets[“main”] + - This is a dictionary containing the values for **all** parameters listed above. Where they have been specified in a config file, or overwritten on the command line, those values will be used. Otherwise the default values listed above will be used +- args.params + - This is a dictionary containing any parameters specified on the command line with -p. + +Within a UFO Ufont object, use font.paramset, since this will include any updates as a result parameter values set in lib.plist. + +In addition to the parameters in the table above, two more read-only parameters can be accessed by scripts - “version” and “copyright” - which give the pysilfont library version and copyright info, based on values in core.py headers. + +### Updating values +Currently only values under Output can be set via scripts, since Backup and Reporting parameters are processed by execute() prior to the script being called. For example: +```python +font.paramset[“precision”] = 9 +``` +would set the precision parameter to 9. + +Note that, whilst reporting _parameters_ can’t be set in scripts, _reporting levels_ can be updated by setting values in the args.logger() object, eg `args.logger.scrlevel = “W”.` + +# Technical + +_Note the details below are probably not needed just for developing scripts..._ + +## Basics +The default for all parameters are set in core.py as part of the parameters() object. Those for **all** pysilfont library modules need to be defined in core.py so that execute() can process command-line arguments without needing information from other modules. + +Parameters are passed to scripts via a parameters() object as args.paramsobj. This contains several parameter sets, with “main” being the standard one for scripts to use since that contains the default parameters updated with those (if any) from the config file then the same for any command-line values. + +Parameters can be accessed from the parameter set by parameter name, eg paramsobj.sets[“main”][“loglevel”]. + +Although parameters are split into classes (eg backup, logging), parameter names need to be unique across all groups to allow simple access by name. + +If logging set set to I or V, changes to parameter values (eg config file values updating default values) are logged. + +There should only be ever a single parameters() object used by a script. + +## Paramobj +In addition to the paramsets, the paramobj also contains +- classes: + - A dictionary keyed on class, returning a list of parameter names in that class +- paramclass: + - A dictionary keyed on parameter name, returning the class of that parameter +- lcase: + - A dictionary keyed on lowercase version of parameter name returning the parameter name +- type: + - A dictionary keyed on parameter name, returning the type of that parameter (eg str, boolean, list) +- listtype: + - For list parameters, a dictionary keyed on parameter name, returning the type of that parameters in the list +- logger: + - The logger object for the script + +## Parameter sets +These serve two purposes: +1. To allow multiple set of parameter values to be used - eg two different fonts might have different values in the lib.plist +1. To keep track of the original sets of parameters (“default”, “config file” and “command line”) if needed. See UFO specific for an example of this need. + +Additional sets can be added with addset() and one set can be updated with values from another using updatewith(), for example, to create the “main” set, the following code is used: +``` +params.addset("main",copyset = "default") # Make a copy of the default set +params.sets["main"].updatewith("config file") # Update with config file values +params.sets["main"].updatewith("command line") # Update with command-line values +``` +## UFO-specific +The parameter set relevant to a UFO font can be accessed by font.paramset, so font.paramset[“loglevel"] would access the loglevel. + +In ufo.py there is code to cope with two complications: +1. If a script is opening multiple fonts, in they could have different lib.plist values so font-specific parameter sets are needed +1. The parameters passed to ufo.py include the “main” set which has already had command-line parameters applied. Any in lib.plist also need to be applied, but can’t simply be applied to “main” since command-line parameters should take precedence over lib.plist ones + +To ensure unique names, the parameter sets are created using the full path name of the UFO. Then font.paramset is set to point to this, so scripts do not need to know the underlying set name. + +To apply the parameter sets updates in the correct order, ufo.py does: + +1. Create a new paramset from any lib parameters present +1. Update this with any command line parameters +1. Create the paramset for the font by copying the “main” paramset +1. Update this with the lib paramset (which has already been updated with command line values in step 2) + +## Adding another parameter or class +If there was a need to add another parameter or class, all that should be needed is to add that to defparams in the \_\_init\_\_() of parameters() in core.py. Ensure the new parameter is case-insensitively unique. + +If a class was Ufont-specific and needed to be supported within lib.plist, then ufo.py would also need updating to handle that similarly to how it now handles outparams and ufometadata. diff --git a/docs/scripts.md b/docs/scripts.md new file mode 100644 index 0000000..0b2f748 --- /dev/null +++ b/docs/scripts.md @@ -0,0 +1,1565 @@ +# Pysilfont commands and scripts + +Below is a table listing all the commands installed by Pysilfont followed by descriptions of each command. + +All these commands work in consistent ways in terms of certain standard options (eg -h for help) and default names for many files - see details in [Pysilfont Documentation](docs.md#standard-command-line-options). + +There are further example scripts supplied with Pysilfont, and some of these are also documented in [examples.md](examples.md) + +## Table of scripts + +| Command | Description | +|--------------------------------------------------|--------------------------------------------------------------------------------------------------------| +| [psfaddanchors](#psfaddanchors) | Read anchor data from XML file and apply to UFO | +| [psfbuildcomp](#psfbuildcomp) | Add composite glyphs to UFO based on a Composite Definitions file | +| [psfbuildcompgc](#psfbuildcompgc) | Add composite glyphs to UFO using glyphConstruction based on a CD file | +| [psfbuildfea](#psfbuildfea) | Compile a feature (.fea) file against an existing input TTF | +| [psfchangegdlnames](#psfchangegdlnames) | Change graphite names within GDL based on mappings files | +| [psfchangettfglyphnames](#psfchangettfglyphnames) | Change glyph names in a ttf from working names to production names | +| [psfcheckbasicchars](#psfcheckbasicchars) | Check UFO for glyphs that represent recommended basic characters | +| [psfcheckclassorders](#psfcheckclassorders) | Verify classes defined in xml have correct ordering where needed | +| [psfcheckftml](#psfcheckftml) | Check ftml files for structural integrity | +| [psfcheckglyphinventory](#psfcheckglyphinventory) | Warn for differences in glyph inventory and encoding between UFO and input file (e.g., glyph_data.csv) | +| [psfcheckinterpolatable](#psfcheckinterpolatable) | Check UFOs in a designspace file are compatible with interpolation | +| [psfcheckproject](#psfcheckproject) | Check UFOs in designspace files have consistent glyph inventory & unicode values | +| [psfcompdef2xml](#psfcompdef2xml) | Convert composite definition file to XML format | +| [psfcompressgr](#psfcompressgr) | Compress Graphite tables in a ttf font | +| [psfcopyglyphs](#psfcopyglyphs) | Copy glyphs from one UFO to another with optional scale and rename | +| [psfcopymeta](#psfcopymeta) | Copy basic metadata from one UFO to another, for fonts in related families | +| [psfcreateinstances](#psfcreateinstances) | Create one or more instance UFOs from one or more designspace files | +| [psfcsv2comp](#psfcsv2comp) | Create composite definition file from csv | +| [psfdeflang](#psfdeflang) | Changes default language behaviour in a font | +| [psfdeleteglyphs](#psfdeleteglyphs) | Deletes glyphs from a UFO based on a list | +| [psfdupglyphs](#psfdupglyphs) | Duplicates glyphs in a UFO based on a csv definition | +| [psfexportanchors](#psfexportanchors) | Export UFO anchor data to a separate XML file | +| [psfexportmarkcolors](#psfexportmarkcolors) | Export csv of mark colors | +| [psfexportpsnames](#psfexportpsnames) | Export a map of glyph name to PS name to a csv file | +| [psfexportunicodes](#psfexportunicodes) | Export a map of glyph name to unicode value to a csv file | +| [psffixffglifs](#psffixffglifs) | Make changes needed to a UFO following processing by FontForge | +| [psffixfontlab](#psffixfontlab) | Make changes needed to a UFO following processing by FontLab | +| [psfftml2TThtml](#psfftml2tthtml) | Convert FTML document to html and fonts for testing TypeTuner | +| [psfftml2odt](#psfftml2odt) | Create a LibreOffice Writer file from an FTML test description | +| [psfgetglyphnames](#psfgetglyphnames) | Create a file of glyphs to import from a list of characters to import | +| [psfglyphs2ufo](#psfglyphs2ufo) | Export all the masters in a .glyphs file to UFOs | +| [psfmakedeprecated](#psfmakedeprecated) | Creates deprecated versions of glyphs | +| [psfmakefea](#psfmakefea) | Make a features file base on input UFO or AP database | +| [psfmakescaledshifted](#psfmakescaledshifted) | Creates scaled and shifted versions of glyphs | +| [psfmakewoffmetadata](#psfmakewoffmetadata) | Make the WOFF metadata xml file based on input UFO | +| [psfnormalize](#psfnormalize) | Normalize a UFO and optionally converts it between UFO2 and UFO3 versions | +| [psfremovegliflibkeys](#psfremovegliflibkeys) | Remove keys from glif lib entries | +| [psfrenameglyphs](#psfrenameglyphs) | Within a UFO and class definition, assign new working names to glyphs based on csv input file | +| [psfrunfbchecks](#psfrunfbchecks) | Run Font Bakery checks using a standard profile with option to specify an alternative profile | +| [psfsetassocfeat](#psfsetassocfeat) | Add associate feature info to glif lib based on a csv file | +| [psfsetassocuids](#psfsetassocuids) | Add associate UID info to glif lib based on a csv file | +| [psfsetdummydsig](#psfsetdummydsig) | Add a dummy DSIG table into a TTF font | +| [psfsetglyphdata](#psfsetglyphdata) | Update and/or sort glyph_data.csv based on input file(s) | +| [psfsetglyphorder](#psfsetglyphorder) | Load glyph order data into public.glyphOrder based on a text file | +| [psfsetkeys](#psfsetkeys) | Set key(s) with given value(s) in a UFO p-list file | +| [psfsetmarkcolors](#psfsetmarkcolors) | Set mark colors based on csv file | +| [psfsetpsnames](#psfsetpsnames) | Add public.postscriptname to glif lib based on a csv file | +| [psfsetunicodes](#psfsetunicodes) | Set unicode values for a glif based on a csv file | +| [psfsetversion](#psfsetversion) | Change all the version-related info in a UFO's fontinfo.plist | +| [psfshownames](#psfshownames) | Display name fields and other bits for linking fonts into families | +| [psfsubset](#psfsubset) | Create a subset of an existing UFO | +| [psfsyncmasters](#psfsyncmasters) | Sync metadata in master UFO files based on a Designspace file | +| [psfsyncmeta](#psfsyncmeta) | Copy basic metadata from one member of a font family to other family members | +| [psftuneraliases](#psftuneraliases) | Merge alias information into TypeTuner feature xml file | +| [psfufo2glyphs](#psfufo2glyphs) | Generate a glyphs files from a designspace file and UFO(s) | +| [psfufo2ttf](#psfufo2ttf) | Generate a ttf file without OpenType tables from a UFO | +| [psfversion](#psfversion) | Display version info for pysilfont and dependencies | +| [psfpreflightversion](#psfpreflightversion) | Display version info for pysilfont and dependencies but only for preflight | +| [psfwoffit](#psfwoffit) | Convert between ttf, woff, and woff2 | +| [psfxml2compdef](#psfxml2compdef) | Convert composite definition file from XML format | + +--- + +#### psfaddanchors +Usage: **`psfaddanchors [-i ANCHORINFO] [-a] [-d] [-r {X,S,E,P,W,I,V}] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +_This section is Work In Progress!_ + +Read anchor data from XML file and apply to UFO + +optional arguments: + +``` + -i ANCHORINFO, --anchorinfo ANCHORINFO + XML file with anchor data + -d, --delete Delete APs from a glyph before adding + -a, --analysis Analysis only; no output font generated + -r {X,S,E,P,W,I,V}, --report {X,S,E,P,W,I,V} + Set reporting level for log fileUpdate and/or sort glyph_data.csv based on input file(s) +``` + +--- + +#### psfbuildfea +Usage: **`psfbuildfea -o output.ttf [-m map.txt] [-v] input.fea input.ttf`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Uses fontTools to compile a feature (.fea) file against an existing input TTF, optionally creating a lookup map. + +required arguments: + +``` + -o output.ttf, --output output.ttf Output file to create + input.fea Source features file + input.ttf Source ttf +``` + +optional arguments: + +``` + -m map.txt, --lookupmap map.txt Mapping file to create + -v, --verbose repeat to increase verbosity +``` + +If `-m` parameter is supplied, the designated file will be created listing, in alphabetical order by name, each OpenType lookup name, its table (GSUB or GPOS) and its feature index. For example: +``` +AlefMark2BelowAfterLam,GPOS,13 +AyahAlternates,GSUB,46 +CommaAlternates,GSUB,48 +``` + +--- + +#### psfbuildcomp +Usage: **`psfbuildcomp [-i CDFILE] [-a] [-c] [--colors] [-f] [-n] [--remove] [--preserve] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Creates or updates composite glyphs in a UFO based on an external text file of definitions. The syntax for these definitions is described in [composite.md](composite.md). + +Example usage: + +``` +# add composites to font (making backup first) +psfbuildcomp -i composites.txt font.ufo + +# add composites even for glyphs that have outlines, and write to a new font +psfbuildcomp -i comps.txt -f -r V Andika-BoldItalic.ufo new.ufo + +# report only, with no change to font +psfbuildcomp -i comps.txt -a -r I font.ufo + +# remove unwanted anchors 'above' and 'below', including '_' versions: +psfbuildcomp -i comps.txt --remove "_?(above|below)" font.ufo + +# also preserve 'diaA' and 'diaB' on composites that exist but are being replaced: +psfbuildcomp -i comps.txt --remove "_?(above|below)" --preserve "dia[AB]" font.ufo + +``` + +optional arguments: + +``` + -i CDFILE, --cdfile CDFILE + Composite Definitions input file + -a, --analysis Analysis only; no output font generated + -c, --color Set the markColor of of generated glyphs dark green + --colors Set the markColor of the generated glyphs based on color(s) supplied + (more details below) + -f, --force Force overwrite of glyphs having outlines + -n, --noflatten Do not flatten component references + --remove REMOVE a regex matching anchor names that should always be + removed from generated composite glyphs + --preserve PRESERVE a regex matching anchor names that, if present in + glyphs about to be replaced, should not be overwritten +``` + +Using --colors, three colors can be supplied: + +- The first for glyphs already in the font which are unchanged in this run of the script +- The second for glyphs already in the font which are updated +- The third for new glyphs added + +If just one color is supplied, this is used for all three uses above + +If two colors are supplied, the second is also used for new glyphs + +Colors can be specified as described in [Specifying colors on the command line](#specifying-colors-on-the-command-line) + +--- +#### psfbuildcompgc +Usage: **`psfbuildcompgc [-i CDFILE] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Creates or updates composite glyphs in a UFO based on an external text file of definitions. The syntax for these definitions *not* the same as that described in [composite.md](composite.md). It uses the [GlyphConstruction syntax](https://github.com/typemytype/GlyphConstruction). + +Example usage: + +``` +psfbuildcompgc -i composites.txt font.ufo +``` + +optional arguments: + +``` + -i CDFILE, --cdfile CDFILE + Composite Definitions input file +``` + +--- +#### psfchangegdlnames +Usage: **`psfchangegdlnames [-n NAMES] [--names2 [NAMES2]] [--psnames PSNAMES] input [output]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Changes the graphite names within GDL files(s) based on mappings file(s). It can work on an individual file or on all the gdl/gdh files within a folder. It also updates postscript names in postscript() statements + +Two mappings files are required (NAMES and PSNAMES). Optionally a second GDL names mapping file, NAMES2 can be supplied. + +The mapping files are csv files of the format `"old name,new name"`. It logs if any graphite names are in the GDL but not found in the mapping files. + +Example usage: + +``` +psfchangegdlnames -n gdlmap.csv --psnames psnames.csv source/graphite +``` +will update all the .gdl and.gdh files within the source/graphite folder. + +--- +#### psfchangettfglyphnames +Usage: **`psfchangettfglyphnames iufo ittf ottf`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Used to change the glyph names in a ttf from working names to production names, typically as the last step in a build sequence. + +The name map is obtained from the `public.postscriptNames` attribute in the input UFO and then applied to the input ttf to create the output ttf. + +Example usage: + +``` +psfchangettfglyphnames source/Harmattan-Regular.ufo results/in.ttf results/out.ttf +``` + +--- +#### psfcheckbasicchars +Usage: **`psfcheckbasicchars [-r] [-s] ufo`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Used to check a UFO for the presence of glyphs that represent the characters in the list of +[Recommended characters for Non-Roman fonts](https://github.com/silnrsi/pysilfont/blob/master/lib/silfont/data/required_chars.csv). +Any missing characters are noted in the resulting log file along with the recommended AGL glyph name. + +By default only characters needed for all fonts (both LTR and RTL) will be checked.\ +To also check for characters that only RTL fonts need, use the -r option.\ +To include characters that are in SIL's PUA block, use the -s option. + +Example usage: + +``` +psfcheckbasicchars Nokyung-Regular.ufo +``` + +There is more documentation about the character list [here](https://github.com/silnrsi/pysilfont/blob/master/lib/silfont/data/required_chars.md) +and additional information can be shown on screen or in the log file by increasing the log level to I (-p scrlevel=i or -p loglevel=i) + +--- +#### psfcheckclassorders +usage: **`psfcheckclassorders [--gname GNAME] [--sort HEADER] [classes] [glyphdata]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Verify classes defined in xml have correct ordering where needed. + +Looks for comment lines in the `classes` file that match the string: +``` + *NEXT n CLASSES MUST MATCH* +``` +where `n` is the number of upcoming class definitions that must result in the +same glyph alignment when glyph names are sorted by TTF order. + +``` +optional arguments: + classes Class definition file in XML format (default `classes.xml`) + glyphdata Glyph info csv file (default `glyph_data.csv`) + --gname GNAME Column header for glyph name (default `glyph_name`) + --sort HEADER Column header for sort order (default `sort_final`) +``` +#### Notes + +Classes defined in xml format (typically `source/classes.xml`) can be accessed by both Graphite and OpenType code. For historical reasons there is a difference in the way they are processed: for Graphite (only), the members of the classes are re-ordered based on the glyphIDs in the ttf. + +For classes used just for rule contexts, glyph order doesn't matter. But for classes used for n-to-n substitutions, order *does* matter and the classes have to be "aligned". + +Based on the sort order information extracted from the `glyphdata` file, this tool examines specially-marked groups of class definitions from the `classes` file to determine if they remain aligned after classes are reordered, and issues error messages if not. Here is an example identifying a set of three classes that must align: +``` + <!-- *NEXT 3 CLASSES MUST MATCH* --> + + <class name='Damma'> + damma-ar shadda_damma-ar hamza_damma-ar + </class> + + <class name='Damma_filled'> + damma-ar.filled shadda_damma-ar.filled hamza_damma-ar.filled + </class> + + <class name='Damma_short'> + damma-ar.short shadda_damma-ar.short hamza_damma-ar.short + </class> + ``` + +Note that the Graphite workflow extracts glyph order from the ttf file, but `psfcheckclassorders` gets it from `glyphdata` argument; there is, therefore, an assumption that the glyph order indicated in `glyphdata` actually matches that in the ttf file. + +`psfcheckclassorders` will also issue warning a warning message if there are glyphs named in the `classes` file which are not included in the `glyphdata` file. While this is [intentionally] not an error for either Graphite or OpenType (relevant tools simply ignore such missing glyphs), it may be helpful in catching typos that result in class miss-alignment and therefore bugs. By default warning messages are sent to the log file; +use `-p scrlevel=W` to also route them to the terminal. + +--- +#### psfcheckftml +usage: **`psfcheckftml [inftml]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Test structural integrity of one or more ftml files + +Assumes ftml files have already validated against [`FTML.dtd`](https://github.com/silnrsi/ftml/blob/master/FTML.dtd), for example by using: + +``` xmllint --noout --dtdvalid FTML.dtd inftml.ftml``` + +`psfcheckftml` verifies that: + - `silfont.ftml` can parse the file + - every `stylename` is defined the `<styles>` list + +``` +positional arguments: + inftml Input ftml filename pattern (default: *.ftml) + +other arguments: + -h, --help show this help message and exit + -l LOG, --log LOG Log file + -p PARAMS, --params PARAMS + Other parameters - see parameters.md for details + -q, --quiet Quiet mode - only display severe errors +``` + +--- +#### psfcheckglyphinventory +Usage: **`psfcheckglyphinventory [--indent n] [-i input] ifont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +`psfcheckglyphinventory` compares and warns for differences in glyph inventory and encoding between UFO and input file (e.g., glyph_data.csv). +input file can be: +- simple text file with one glyph name per line +- csv file with headers, using headers `glyph_name` and, if present, `USV` + +required arguments: + +``` + ifont input UFO +``` + +optional arguments: +``` + -i INPUT, --input INPUT + input file, default is glyph_data.csv in current directory + -indent n number of spaces to indent output lists (default 10) +``` + +--- +#### psfcheckinterpolatable +Usage: **`psfcheckinterpolatable designspace`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Check that the UFOs listed in the designspace file are interpolatable. +When there are more than two UFOs in the designspace files they are checked in pairs. For each pair: + - The glyph inventories of the UFOs are compared and any discrepancies reported. + - For each glyph that is in both UFOs, they are tested with fontParts isCompatible() function which checks various items - for example the number and direction of contours. + +--- +#### psfcheckproject +Usage: **`psfcheckproject designspace(s)`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Check that the UFOs listed in the designspace file(s) have consistent glyph inventories and glyph unicode values. + +Example usage: +``` +psfcheckproject source/*.designspace +``` + +Multiple designspace files maybe supplied and wildcards can be used. + +Further project-wide checks may be added to this script at a later date. + + +---- +#### psfcompdef2xml +Usage: **`psfcompdef2xml [-p PARAMS] input output log`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Convert composite definition file to XML format + +_This section is Work In Progress!_ + + input Input file of CD in single line format + output Output file of CD in XML format + log Log file + + -p PARAMS, --params PARAMS + XML formatting parameters: indentFirst, indentIncr, + attOrder + + Defaults + output \_out.xml + log \_log.txt + +--- +#### psfcompressgr +Usage: **`psfcompressgr ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Compress Graphite tables in a font + +--- +#### psfcopyglyphs +Usage: **`psfcopyglyphs [-i INPUT] -s SOURCE ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Copy selected glyphs from a source UFO to a target UFO. + +required arguments: + +``` + ifont Target font into which glyphs will be copied + -s SOURCE, --source SOURCE + Font to get glyphs from +``` + +optional arguments: +``` + ofont output font to create instead of rewriting ifont + -i INPUT, --input INPUT + CSV file identifying glyphs to copy + -n NAME, --name NAME Include glyph named NAME + -f, --force Overwrite existing glyphs in the target font + --scale SCALE Scale glyphs by this factor + --rename COLHEADER Names column in CSV containing new names for glyphs + --unicode COLHEADER Names column in CSV containing USVs to assign to glyphs +``` + +Glyphs to be copied are specified by the `INPUT` CSV file and/or on the command line. + +When provided, if the CSV file has only one column, a column header is not needed and each line names one glyph to be copied. + +If the CSV file has more than one column, then it must have headers, including at least: + +- `glyph_name` contains the name of the glyph in the SOURCE that is to be copied. + +If `--rename` parameter is supplied, it identifies the column that will provide a new name for the glyph in the target font. For any particular glyph, if this column is empty then the glyph is not renamed. + +If `--unicode` parameter is supplied, it identifies the column that provides an optional Unicode Scalar Value (USV) for the glyph in the target font. For any particular glyph, if this column is empty then the glyph will not be encoded. + +Glyphs to be copied can also be specified on the command line via one or more `--name` parameters. Glyphs specified in this way will not be renamed or encoded. + +If any glyph identified by the CSV or `--name` parameter already exists in the target font, it will not be overwritten unless the `--force` parameter is supplied. + +If any glyph being copied is a composite glyph, then its components are also copied. In the case that a component has the same name as a glyph already in the font, the component is renamed by appending `.copyN` (where N is 1, 2, 3, etc.) before being copied. + +Limitations: + +- At present, the postscript glyph names in the target font are left unchanged and may therefore be inaccurate. Use `psfsetpsnames` if needed. + +--- +#### psfcopymeta +Usage: **`psfcopymeta [-r] fromfont tofont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This copies selected fontlist.plist and lib.plist metadata (eg copyright, openTypeNameVersion, decender) between fonts in different (related) families. + +It is usually run against the master (regular) font in each family then data synced within family afterwards using [psfsyncmeta](#psfsyncmeta). + +Example usage: + +``` +psfcopymeta GentiumPlus-Regular.ufo GentiumBookPlus-Bold +``` + +If run with -r or \-\-reportonly it just reports what values would be updated. + +Look in psfcopymeta.py for a full list of metadata copied. Note that only fontinfo.plist and lib.plist are updated; the target font is not normalized. + +Also psfcopymeta does not use Pysilfont's backup mechanism for fonts. + +--- +#### psfcreateinstances +Usage: + +**`psfcreateinstances -f [--roundInstances] designspace_file_or_folder`** + +**`psfcreateinstances [-i INSTANCENAME] [-a INSTANCEATTR] [-v INSTANCEVAL] [-o OUTPUT] +[--forceInterpolation] [--roundInstances] [--weightfix|-W] designspace_file`** + + +Create one or more instance UFOs from one or more designspace files. + +There are two modes of operation, differentiated by the `-f` (folder) option: + +When `-f` is specified: +- the final parameter can be either: + - a single designspace file + - a folder, in which case all designspace files within the folder are processed. +- all instances specified in the designspace file(s) are created. +- interpolation is always done, even when the designspace coordinates of an instance match a master. + +Omitting the `-f` requires that the final parameter be a designspace file (not a folder) but gives more control over instance creation, as follows: + +- Specific instance(s) to be created can be identified by either: + - instance name, specified by `-i`, or + - a point on one of the defined axes, specified by `-a` and `-v`. If more than one instance matches this axis value, all are built. +- The default location for the generated UFO(s) can be changed using `-o` option to specify a path to be prefixed to that specified in the designspace. +- In cases where the designspace coordinates of an instance match a master, glyphs will be copied rather than interpolated, which is useful for masters that do not have compatible glyph designs and thus cannot be interpolated. This behavior can be overridden using the `--forceInterpolation` option. + +Whenever interpolation is done, the calculations can result in non-integer values within the instance UFOs. The `--roundInstances` option will apply integer rounding to all such values. + +Instance weights are set based on the axes mapping in the designspace file. +If the weight is missing from the axes mapping a dummy value of 399 is set. + +When `--weightfix` (or `-W`) is provided, instance weights are set to either 700 (Bold) or 400 (Regular) based on +whether or not the `stylemapstylename` instance attribute begins with `bold`. + +--- +#### psfcsv2comp +Usage: **`psfcsv2comp [-i INPUT] [--gname GNAME] [--base BASE] [--anchors ANCHORS] [--usv USV] output.txt`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Create a composite definitions file based on data extracted from a csv file. + +The INPUT csv file must have column headers and must include columns for the following data: +- the name of the composite glyph +- the name of the base glyph used to create the composites +- one column for each possible anchor to which a component can be attached. The column headers identify the name of the anchor to be used, and the column content names the glyph (if any) to be attached at that anchor. + +Optionally, another column can be used to specify the USV (codepoint) for the composite. + +Command-line options: + +- INPUT: Name of input csv file (default `glyph_data.csv`) +- GNAME: the column header for the column that contains the name of the composite glyph (default `gname`) +- BASE: the column header for the column that contains the base of the composites (default `base`) +- ANCHORS: comma-separated list of column headers naming the attachment points (default `above,below`). +- USV: the column header for the column that contains hexadecimal USV + +**Limitations:** At present, this tool supports only a small subset of the capabilities of composite definition syntax. Note, in particular, that it assumes all components are attached to the _base_ rather than the _previous glyph_. + +--- +#### psfdeflang +Usage: **`psfdeflang -L lang infont [outfont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This changes the default language behaviour of a .ttf font from its current default to that of the language specified. It supports both OpenType and Graphite tables. + +For example this command creates a new font which by default has Khamti behaviour: + +``` +psfdeflang -L kht Padauk-Regular.ttf Padauk_kht-Regular.ttf +``` + +--- +#### psfdeleteglyphs +Usage: **`psfdeleteglyphs [-i DELETELIST] [--reverse] infont [outfont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This deletes glyphs in a UFO based on an external file with one glyphname per line. +The `--reverse` option will instead delete all glyphs in the UFO that are not in the list. + +It only deletes glyphs that do exist in the default layer, but for such glyphs they are also deleted from other layers, as well as in groups.plist and kerning.plist. +Kern groups based on the glyph name, ie public.kern1._glyphname_ or public.kern2._glyphname_ are also deleted. + +It does not analyze composites, so be careful not to delete glyphs that are referenced as components in other glyphs. + +The following example will delete all glyphs that are _not_ listed in `keepthese.txt`: + +``` +psfdeleteglyphs Andika-Regular.ufo -i keepthese.txt --reverse +``` + +--- +#### psfdupglyphs +Usage: **`psfdupglyphs [-i INPUT] [--reverse] infont [outfont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This duplicates glyphs in a UFO based on a csv definition: source,target. It duplicates everything except unicodes. + +Example usage: + +``` +psfdupglyphs Andika-Regular.ufo -i dup.csv +``` + +--- +#### psfexportanchors +Usage: **`psfexportanchors [-r {X,S,E,P,W,I,V}] [-g] [-s] ifont [output]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This exports anchor data from a UFO font to an XML file. (An "anchor" is also called an "attachment point" which is sometimes abbreviated to "AP".) + +Example that exports the anchors contained in the UFO font `CharisSIL-Regular.ufo`, sorts the resulting glyph elements by public.glyphOrder rather than glyph ID (GID), and writes them to an XML file `CharisSIL-Regular_ap.xml`. + +``` +psfexportanchors -s font-charis/source/CharisSIL-Regular.ufo CharisSIL-Regular_ap.xml +``` + +If the command line includes + +- -g, then the GID attribute will be present in the glyph element. +- -s, then the glyph elements will be sorted by public.glyphOrder in lib.plist (rather than by GID attribute). +- -u, then the UID attribute will include a "U+" prefix + +--- +#### psfexportmarkcolors +Usage: **`psfexportmarkcolors [-c COLOR] [-n] [-o OUTPUT] [--nocomments] ifont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This exports a mapping of glyph name to cell mark color to a csv file, format "glyphname,colordef". +Colordef is exported as a double-quoted string according to the [color definition standard](https://unifiedfontobject.org/versions/ufo3/conventions/#colors). It includes comments at the start saying when it was run etc unless --nocomments is specified. The csv produced will include all glyphs, whether or not they have a color definition. + +In some cases (see options below) colors can be reported or referred to by text names as in "g_purple". See [Specifying colors on the command line](#specifying-colors-on-the-command-line) + +If the command line includes + +- -c COLOR, then the script will instead produce a list of glyph names, one per line, of glyphs that match that single color. +- -n, then the csv file will report colors using text names (see above) rather than using numerical definitions. +If there is no name that matches a particular color definition then it will be exported numerically. + +Example that exports a csv file (glyphname, colordef) listing every glyph and its color as in `LtnSmA,"0.5,0.09,0.79,1"`: + +``` +psfexportmarkcolors Andika-Regular.ufo -o markcolors.csv +``` + +Example that exports a list of glyphs that are colored purple: + +``` +psfexportmarkcolors Andika-Regular.ufo -o glyphlist.txt -c "g_purple" +``` + +--- +#### psfexportpsnames +Usage: **`psfexportpsnames [-o OUTPUT] [--nocomments] ifont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Export a mapping of glyph name to postscript name to a csv file, format "glyphname,postscriptname" + +It includes comments at the start saying when it was run etc unless \-\-nocomments is specified + +--- +#### psfexportunicodes +Usage: **`psfexportunicodes [-o OUTPUT] [--nocomments] [--allglyphs] ifont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Export a mapping of glyph name to unicode to a csv file, format "glyphname,unicode" for glyphs that have a defined unicode. _Note: multiple-encoded glyphs will be ignored._ + +It includes comments at the start saying when it was run etc unless \-\-nocomments is specified + +A complete list of glyphs (both encoded and unencoded) can be generated with \-\-allglyphs. + +--- +#### psffixffglifs +Usage: **`psffixffglifs ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Make changes needed to a UFO following processing by FontForge. Currently FontForge copies advance and unicode fields from the default layer to the background layer; this script removes all advance and unicode fields from the background layer. + +Note that other changes are reversed by standard [normalization](docs.md#Normalization) and more by using pysilfont's standard check&fix system, so running psffixffglyphs with check&fix may be useful: + +``` +psffixffglifs font.ufo -p checkfix=y +``` + +#### psffixfontlab +Usage: **`psffixfontlab ifont` ** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Make changes needed to a UFO following processing by FontLab, including restoring information from the backup of the UFO. + +When exporting from Fontlab, the option _Existing Font Files:_ must be set to _rename_ in order to create the backup needed. +If there are multiple backup files, the oldest will be used on the assumption that several exports have been run since psffixfontlab was last used. +So it is important that any old backups are deleted before re-editing a font. + +The changes made by `psffixfontlab` include: +- Restoring groups.plist, kerning.plist and any layerinfo.plist files from the backups +- Deleting various keys from fontinfo.list and lib.plist +- Restoring guidelines in fontinfo.plist and plublic.glyphOrder from lib.plist + +Sample usage +``` +psffixfontlab font.ufo -p checkfix=None +``` +Notes +- The above example has checkfix=None, since otherwise it will report errors and warnings prior to the script fixing them +- Pysilfont's normal backup mechanism for fonts is not used. + +--- +#### psfftml2TThtml +Usage: **`usage: psfftml2TThtml --ftml FTML --xslt XSLT ttfont map`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Used for testing TypeTuner code, this tool: +- for each FTML document supplied: + - based on the styles therein, creates TypeTuned fonts that should do the same thing + - emits an HTML document that shows the FTML data rendered using both: + - The original font (using font features or language tags) + - The corresponding TypeTuned font + + +positional arguments: +``` + ttfont Input Tunable TTF file + map Feature mapping CSV file +``` + +The `map` parameter must be a CSV that maps names of font feature tags, font feature values and language tags used in the FTML document(s) to the corresponding TypeTuner feature names and values. For example the following CSV file: + +``` +# Language tag mappings: +# lang=langtag,TT feat,value +lang=sd,Language,Sindhi +lang=ur,Language,Urdu + +# Feature tag mappings: +# OT feature,TT feat,default,value 1,value 2,... +cv48,Heh,Standard,Sindhi-style,Urdu-style +``` +maps: +- langtag `sd` to the TypeTuner feature named `Language` with value `Sindhi` +- langtag `ur` to the TypeTuner feature named `Language` with value `Urdu` +- font feature `cv48` to the TypeTuner feature named `Heh`, with following TypeTuner value names: + - cv48 value `0` to `Standard` + - cv48 value `1` to `Sindhi-style` + - cv48 value `2` to `Urdu-style` + +other arguments: +``` + -h, --help show this help message and exit + -o OUTPUTDIR, --outputdir OUTPUTDIR + Output directory, default: tests/typetuner + --ftml FTML ftml file(s) to process. Can be used multiple + times and can contain filename patterns. + --xsl XSL standard xsl file. Default: ../tools/ftml.xsl + --norebuild assume existing fonts are good + -l LOG, --log LOG Log file + -p PARAMS, --params PARAMS + Other parameters - see parameters.md for details + -q, --quiet Quiet mode - only display severe errors +``` + +For reliability, the program re-builds each needed font even if that font was built during a previous run of the program. For debugging, specifying `--norebuild` will speed up the program by assuming previously built fonts are usable. + +--- +#### psfftml2odt +Usage: **`psfftml2odt [-f FONT] input [output]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This creates a LibreOffice writer document based on input test data in [Font Test Markup Language](https://github.com/silnrsi/ftml) format and font information specified with command line parameters. + +Example that uses FTML input contained in the file `test-ss.xml` and creates a LibreOffice writer document named `test-ss.odt`. There will be two columns in the output document, one for the installed font `Andika New Basic` and one for the font contained in the file `AndikaNewBasic-Regular.ttf`. (This compares a newly built font with an installed reference.) + +``` +psfftml2odt -f "Andika New Basic" -f "AndikaNewBasic-Regular.ttf" test-ss.xml test-ss.odt +``` + +If the font specified with the -f parameter contains a '.' it is assumed to be a file name, otherwise it is assumed to be the name of an installed font. In the former case, the font is embedded in the .odt document, in the latter case the font is expected to be installed on the machine that views the .odt document. + +--- +#### psfgetglyphnames +Usage: **`psfgetglyphnames [-i INPUT] [-a AGLFN] ifont glyphs`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Given a list of characters to import in INPUT +(format is one character per line, using four or more hex digits) +and a source UFO infont (probably the source of Latin glyphs for a non-roman font), +create a list of glyphs to import for use with the +[psfcopyglyphs](#psfcopyglyphs) tool. + +The AGLFN option will rename glyphs on import if found in the +Adobe Glyph List For New Fonts (AGLFN). +The format for this file is the same as the AGLFN from Adobe, +except that the delimiter is a comma, not a semi-colon. + +--- +#### psfglyphs2ufo +Usage: **`psfglyphs2ufo [--nofixes] [--nofea] [--preservefea] [--restore] fontfile.glyphs masterdir`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Exports one UFO file per master found in the fontfile.glyphs file, and places it in the directory specified as masterdir. + +In a round-trip ufo -> glyphs -> ufo there is currently there is some data loss in the standard glyphs +-> ufo conversion, so (unless `--nofixes` is set) the script fixes some data and restores some fields from the original ufos if they are present in the masterdir. + +Additional fields to restore can be added using `-r, --restore`. This will restore the fields listed if found in either fontinfo.plist or lib.plist + +Currently features.fea does not round-trip successfully, so `--nofea` can be used to suppress the production of a features.fea file. + +To leave any features.fea files in existing UFOs untouched use `--preservefea` + +Example usage: + +``` +psfglyphs2ufo CharisSIL-RB.glyphs masterufos +psfglyphs2ufo CharisSIL-RB.glyphs masterufos -r key1,key2 +``` + +If this Glyphs file contains two masters, Regular and Bold, then it will export a UFO for each into a 'masterufos' directory. To have the fonts exported to the current directory, give it a blank directory name: + +``` +psfglyphs2ufo CharisSIL-RB.glyphs "" +``` +--- +#### psfmakedeprecated +Usage: **`psfmakedeprecated [-i INPUT] [--reverse] infont [outfont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Creates deprecated versions of glyphs: takes the specified glyph and creates a duplicate with an additional box surrounding it so that it becomes reversed, and assigns a new unicode encoding to it. +Input is a csv with three fields: original,new,unicode + +Example usage: + +``` +psfmakedeprecated Andika-Regular.ufo -i deprecate.csv +``` + +--- +#### psfmakefea +Usage: **`usage: psfmakefea [-i INPUT] [-o OUTPUT] [-c CLASSFILE] + [--classprops] [--omitaps OMITAPS] infile`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Creates OUTPUT feature (FEA) file by merging the INPUT feature (FEA or FEAX) file with information gleaned from an input UFO or [attachment point (AP)](https://metacpan.org/pod/distribution/Font-TTF-Scripts/scripts/ttfbuilder#Attachment-Points) xml file. For more information about FEAX see [Fea Extensions](feaextensions.md) documentation. + +required arguments: + +``` + infile UFO or AP xml files + INPUT FEA or FEAX input file +``` + +optional arguments: +``` + OUTPUT name of FEA file to create (if not supplied, only error checking is done) + CLASSFILE name of xml class definition file + --classprops if specified, class properties will be read from CLASSFILE + OMITAPS comma-separated list of attachment points to ignore when creating classes +``` + +--- +#### psfmakescaledshifted +Usage: **`psfmakescaledshifted [-c] [--color COLOR] -i INPUT -t TRANSFORM infont [outfont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Creates scaled and shifted versions of glyphs: takes the specified glyph and creates a duplicate that is scaled and shifted according to the specified transform, and assigns a new unicode encoding to it. Optional -c marks cells of generated glyphs (dark blue). + +Input is a csv with three fields: *original,new,unicode*. + +Transform takes two types of input: + +- a string of the form "(xx, xy, yx, yy, x, y)" where xx = amount to scale horizontally, yy = amount to scale vertically, x = amount to shift horizontally, y = amount to shift vertically. xy and yx are generally not used and remain 0. +- the name of a specific type of transform defined in the UFO lib.plist *org.sil.lcg.transforms* key, such as superscript. Example: + +``` +<key>org.sil.lcg.transforms</key> +<dict> + <key>superscript</key> + <dict> + <key>adjustMetrics</key> + <integer>0</integer> + <key>scaleX</key> + <real>0.66</real> + <key>scaleY</key> + <real>0.6</real> + <key>shiftX</key> + <integer>-125</integer> + <key>shiftY</key> + <integer>-460</integer> + <key>skew</key> + <real>-0.01</real> + </dict> +</dict> +``` + +Note that this second type of input allows for two other parameters: + +- _adjustMetrics_ indicates how much additional space in units should be added to _both_ sides of the glyph. + +- _skew_ indicates how much the glyph should be skewed, with a skew of 1 indicating a 45° skew. The origin for the skew is (0,0). + +There are also two further transformation parameters that can be added to _org.sil.lcg.transforms_ solely for the purpose of documenting post-transformation manual design adjustments. **These are not read or applied by the script. They are only to hold information for the designer.**: + +- _manAdjustX_ indicates how much x-axis weight in units should be manually added to glyphs after the script has been applied. + +- _manAdjustY_ indicates how much y-axis weight in units should be manually added to glyphs after the script has been applied. + +Examples: + +``` +psfmakescaledshifted -i newglyphs.csv DoulosSIL-Regular.ufo -t "(0.72, 0, 0, 0.6, 10, 806)" +``` + +This will take the definitions in newglyphs.csv and create the new glyphs using a transformation that includes x-scale 72%, y-scale 60%, x-shift 10 units, y-shift 806 units. + +``` +psfmakescaledshifted -i newglyphs.csv DoulosSIL-Regular.ufo -t superscript +``` + +This will take the definitions in newglyphs.csv and create the new glyphs using the *superscript* transformation defined in the UFO lib.plist *org.sil.lcg.transforms* key. + +`-c` or `--color COLOR` can be used to set the mark color for the generated glyphs. `-c` sets the color to blue, and with +`--color` the color specified as described in [Specifying colors on the command line](#specifying-colors-on-the-command-line) + +--- +#### psfmakewoffmetadata +Usage: **`psfmakewoffmetadata -n PRIMARYFONTNAME -i ORGID [-f FONTLOG] [-o OUTPUT] [--populateufowoff] fontfile.ufo`** + + +Make the WOFF metadata xml file based on input UFO. If woffMetadataCredits and/or woffMetadataDescription are missing +from the UFO, they will be constructed from FONTLOG - see below + +The primary font name and orgid need to be supplied on the command line. By default it outputs to *primaryfontname*-WOFF-metadata.xml. + +Example: + +``` +psfmakewoffmetadata -n "Nokyung" -i "org.sil.fonts" source/Nokyung-Regular.ufo +``` + +It constructs the information needed from: + +- The supplied primary font name and orgid +- Information within the primary font file + +If it needs to construct the credits and description fields from FONTLOG, that file needs to be formatted according to the +pattern used for most SIL font packages. +The description is created using the contents of the FONTLOG, starting after the "Basic Font Information" header +and finishing before the "Information for Contributors" header (if present) or the "Acknowledgements" header otherwise. +The credits are created from the N:, W:, D: and E: sets of values in the acknowledgements section, though E: is not used. +One credit is created from each set of values and sets should be separated by blank lines. +By default it reads FONTLOG.txt from the folder it is run in. + +If woffMetadataCredits and woffMetadataDescription are missing from the UFO you can use `--populateufowoff` to update +these fields with the values constructed from FONTLOG: +``` + --populateufowoff Add woffMetadataCredits and woffMetadataDescription to UFO if missing +``` +--popoulateufowoff also adds a place-holder url to woffMetadataDescription which should be hand-edited afterwards to a url appropirate to the project. + +--- +#### psfnormalize +Usage: **`psfnormalize [-v VERSION] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This normalizes a UFO font (and optionally converts from one version to another if -v is specified). _Note that most pysilfont scripts automatically output normalized UFOs, so psfnormalize is normally only needed after fonts have been processed by external font tools._ + +Example that normalizes the named font: + +``` +psfnormalize Nokyung-Regular.ufo +``` + +The normalization follows the [default behaviours](docs.md#normalization), but these can be overridden using [custom parameters](parameters.md) + +\-v VERSION can be 2, 3 + +If you are a macOS user, see _pysilfont/actionsosx/README.txt_ to install an action that will enable you to run psfnormalize without using the command line. + +--- +#### psfremovegliflibkeys +Usage: **`psfremovegliflibkeys [-o OFONT] ifont [key [key ...]] [-b [BEGINS [BEGINS ...]]]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This removes the specified key(s) from the lib section of .glif files if they exist. + +``` +psfremovegliflibkeys GentiumPlus-Regular.ufo key1 key2 -b start1 start2 +``` + +This will remove any keys that match key1 or key2 or begin with start1 or start2 + +Note - Special handling for com.schriftgestaltung.Glyphs.originalWidth: +- Due to a glyphsLib bug, advance width is sometimes moved to this key, so if this key is set for deletion + - If advance width is not set in the glif, it is set to com.schriftgestaltung.Glyphs.originalWidth + - com.schriftgestaltung.Glyphs.originalWidth is then deleted + +--- +#### psfrenameglyphs +Usage: **`psfrenameglyphs [--mergecomps] [-c CLASSFILE] -i INPUT ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Assign new working names to glyphs based on csv input file, format "oldname,newname". The algorithm will handle circular rename specifications such as: +``` +glyph1,glyph2 +glyph2,glyph1 +``` +Unless default value for `renameGlyphs` [parameter](parameters.md) is overridden, the .glif filenames in the UFO will also be adjusted. + +This program modifies the glyphs themselves and, if present in lib.plist, the `public.glyphOrder`, `public.postscriptNames`, `com.schriftgestaltung.glyphOrder` and `com.schriftgestaltung.customParameter.GSFont.DisplayStrings` definitions. Any composite glyphs that reference renamed glyphs are adjusted accordingly. + +If groups.plist is present, glyph names in groups are renamed. In addition, groups named public.kern1._glyphname_ or public.kern2._glyphname_ will also be renamed, but group names not matching that pattern are left unchanged. + +If kerning.plist is present, glyph names in kern pairs are changed and kern group names that match the pattern described above are also changed. + +If -c specified, the changes are also made to the named classes definition file. + +When there are multiple layers in the UFO, glyphs will be renamed in all layers providing the glyph is in the default layer. If the glyph is only in non-default layers the glyph will need renaming manually. + +In normal usage, all oldnames and all newnames mentioned in the csv must be unique. + +The `--mergecomps` option enables special processing that allows newnames to occur more than once in the csv, with the result that the first mention is a normal rename while subsequent mentions indicate glyphs that should be deleted but any references updated to the first (renamed) glyph. Any moving anchors (i.e., those whose names start with `_`) on the deleted glyphs will be copied to the first glyph. For example: +``` +dotabove,dot1 # this glyph has _above anchor +dotbelow,dot1 # this glyph has _below anchor +dotcenter,dot1 # this glyph has _center anchor +``` +would cause `dotabove` to be renamed `dot1` while `dotbelow` and `dotabove` would be deleted. Any composite glyphs that reference any of `dotabove`, `dotbelow`, or `dotcenter` will be adjusted to refer to `dot1`. The `_below` anchor from `dotbelow` and the `_center` anchor from `dotcenter` will be copied to `dot1` (overwriting any anchors by the same names). + +Any `--mergecomps` run should be done in a separate run of `psfrenameglyphs` from other renaming, and group and kerning data is not processed on mergecomps runs. + +--- +#### psfrunfbchecks +Usage: **`psfrunfbchecks [--profile PROFILE] [--html HTMLFILE] [--csv CSV] + [--full-lists] [--ttfaudit] fonts [fonts ...]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Run Font Bakery tests using a standard profile and report results in a table on screen. Multiple fonts can be specified, +including using wildcards, but all should be in the same directory. + +Currently this just works for ttf files, and the default profile used will be Pysilfont's ttfchecks.py. + +An alternative profile can be specified with `--profile`. This profile needs to be specifically designed to work with this script - see examples/fbttfchecks.py. This profile amends the behaviour of ttfchecks.py and includes options to change which checks are run and to override the status reported by checks. Project-specific checks can also be added. + +Example use with a project-specific profile: + +``` +psfrunfbchecks --profile fbttfchecks.py results/*.ttf +``` +To see more details of results, use `--html HTMLFILE` to write an html file in the same format that Font Bakery outputs. + +`--csv CSV` creates a csv file with one line per check that is run. This can be used with diff tools to compare results from different runs (or following upgrades to Font Bakery.) + +Pysilfont's standard logging parameters (-p scrlevel and -p loglevel) also change the level of information `psfrunfbchecks` outputs. + +By default Font Bakery truncates long lists of items within check reports. Use `--full-lists` to get full lists of items. + +A special option, `--ttfaudit` compares the list of checks within ttfchecks.py against those in Font Bakery and reports any descrepancies to screen. +It also writes a csv file containing a list of all checks. Usage `psfrunfbchecks --ttfaudit csvfile` + +--- +#### psfsetassocfeat +Usage: **`psfsetassocfeat [-i INPUT] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Add associate feature info to org.sil.assocFeatureValue glif lib based on a csv file, format "glyphname,featurename[,featurevalue]" + +--- +#### psfsetassocuids +Usage: **`psfsetassocuids [-i INPUT] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Add associate UID info to org.sil.assocUIDs in glif lib based on a csv file - could be one value for variant UIDs and multiple for ligatures, format "glyphname,UID[,UID]" + +--- +#### psfsetdummydsig +Usage: **`psfsetdummydsig -i inputfont -o outputfont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Put a dummy DSIG table into a font in TTF format (using fontTools) + +``` +-i [--ifont] inputfont (Input file in TTF format) +-o [--ofont] outputfont (Output file in TTF format) +``` +--- +#### psfsetglyphdata +Usage: **`psfsetglyphdata [-a ADDCSV] [-d DELETIONS] [-s SORTHEADER] [--sortalpha] [-f] glyphdata [outglyphdata]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Update a csv containing glyph data. The updates can be: + - Additions, based on an input csv + - Deletions, based on a text file with a list of glyph names + - Sorting based on a column header + +Any combination of the above can be used. +``` + glyphdata glyph_data csv file to update + outglyphdata Alternative output file name + -a [--addcsv] Records to add to glyphdata + -d [--deletions] Records to delete from glyphdata + -s [--sortheader] Column header to sort by + --sortalpha Use with sortheader to sort alphabetically not numerically + -f, --force When adding, if glyph exists, overwrite existing data +``` + +The input glyph data file must have column headers, and those must include a glyph_name column. + +Headers are optional for ADDCSV: + - If no headers, the rows must contain the same number of fields (and in the same order) as the glyph data file + - If headers are used, then there must be a glyph_name header. + - Headers can be in any order, and if glyph data headers are missing, then those fields will be left empty + +If `--sortheader` is supplied then the file will be sorted by that column; +otherwise the new records will be added to the end of the file. +By default sorting is done numerically, but `--sortalpha` can be used for alphabetic sorting. + +If a glyph already exists, it won't be overwritten unless + - The glyph is in DELETIONS, since they are processed before additions + - `--force` is used to force overwriting of any existing glyphs + +--- +#### psfsetglyphorder +Usage: **`psfsetglyphorder [--gname GNAME] [--header HEADER] [--field FIELD] [-i INPUT] [-x] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +From the INPUT file, load `public.glyphOrder` in lib.plist to control the order of glyphs in generated TTF files. FIELD can be used to specify a different order to load, such as `com.schriftgestaltung.glyphOrder`. + +The input file can be in one of two formats: +- Plain text file with one glyph name per line in the desired order +- csv file with headers using glyph_name and sort_final columns + +With the csv file: +- The glyph names are sorted by the values in the sort_final column, which can be integer or real. HEADER can be used to specify alternate column header to sort_final. Multiple comma-separated values can be used with `--header` and `--field` to update two or more orders in a single command call. +- GNAME can be used to specify column header to use instead of glyph_name. + +By default all entries in the input file are added, even if the glyph is not in the font. The UFO spec allows this so a common list can be used across groups of fonts. Use -x to only add entries for glyphs that exist in the font. + +Example that imports the data based on glyph_name and sort_final columns in the csv, only adding entries for glyphs in the font: +``` +psfsetglyphorder Andika-Regular.ufo -i glyphdata.csv -x +``` +Example that imports the data from the sort_final column to public.glyphorder and from the sort_designer into com.schriftgestaltung.glyphOrder: +``` +psfsetglyphorder Andika-Regular.ufo -i glyphdata.csv --header sort_final,sort_designer --field public.glyphOrder,com.schriftgestaltung.glyphOrder +``` +--- +#### psfsetkeys +Usage: **`psfsetkeys [--plist PLIST] [-i INPUT] [-k KEY] [-v VALUE] [--file FILE] [--filepart FILEPART] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Set keys in a UFO p-list file. +A single key can be set by specifying KEY and one of VALUE, FILE, or FILEPART. +VALUE should be a single line string, FILE and FILEPART should be a filename. +With FILEPART, the contents of the file are read until the first blank line. +This is useful for setting the copyright key from the OFL.txt file. + +Multiple keys can be set using a csv INPUT file, format "key,value". +A filename to read cannot be specified in the csv file. + +By default keys are stored with type string in the UFO. +Values of true or false are converted to type boolean. +Values that can be converted to integer are stored as type integer. + +PLIST selects which p-list to modify. +If not specified defaults to `fontinfo` which means the `fontinfo.plist` file is modified. + +Example: + +Set a key in the file `lib.plist`. +``` +psfsetkeys --plist lib -k com.schriftgestaltung.width -v Regular font.ufo +``` + +--- +#### psfsetmarkcolors +Usage: **`psfsetmarkcolors [-c COLOR] [-i INPUT] [-u] [-x] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This sets the cell mark color of a glyph according to the [color definition standard](https://unifiedfontobject.org/versions/ufo3/conventions/#colors) based on a list of glyph names in INPUT, one glyph name per line. +COLOR may be defined as described in [Specifying colors on the command line](#specifying-colors-on-the-command-line) + +If the command line includes: +- -u, then the INPUT file will be treated as a list of unicode values rather than glyph names, and the color set on any glyph that is encoded with those unicode values. +- -x, then the color definition will be removed altogether. If no INPUT is given all color definitions will be removed from all glyphs. + +Example that sets the cell mark color of all glyphs listed in glyphlist.txt to purple (0.5,0.09,0.79,1): +``` +psfsetmarkcolors Andika-Regular.ufo -i glyphlist.txt -c "0.5,0.09,0.79,1" +``` +Example that sets the cell mark color of all glyphs that have the unicode values listed in unicode.txt to purple (0.5,0.09,0.79,1): +``` +psfsetmarkcolorss Andika-Regular.ufo -u -i unicode.txt -c "g_purple" +``` +Example that sets the cell mark color of all glyphs that have the unicode values listed in unicode.txt to purple (0.5,0.09,0.79,1): +``` +psfsetmarkcolors Andika-Regular.ufo -u -i unicode.txt -c "g_purple" +``` +Example that removes all color definitions from all glyphs: (this is effectively equivalent to `psfremovegliflibkeys Andika-Regular.ufo public.markColor`) +``` +psfsetmarkcolors Andika-Regular.ufo -x +``` + +--- +#### psfsetpsnames +Usage: **`psfsetpsnames [--gname GNAME] [-i INPUT] [-x] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +From the INPUT file, populate `public.postscriptName` in lib.plist to specify final production names for glyphs. + +The input file can be in one of two formats: +- simple csv in form glyphname,postscriptname +- csv file with headers using glyph_name and ps_name columns + +With the csv file, GNAME can be used to specify column header to use instead of glyph_name. + +By default all entries in the input file are added, even if the glyph is not in the font. The UFO spec allows this so a common list can be used across groups of fonts. Use -x to only add entries for glyphs that exist in the font. + +Example usage: +``` +psfsetpsnames Andika-Regular.ufo -i psnames.txt +``` + +--- +#### psfsetunicodes +Usage: **`psfsetunicodes [-i INPUT] ifont [ofont]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Set the unicodes of glyphs in a font based on a csv file with format "glyphname,UID [,UID2 [,UID3]]". Unicode values must be hex digits with no prefix. + +Up to 3 UIDs can be specified per glyph. + +Any existing Unicode values for the glyph will be removed, and any other glyph that has that same Unicode value will have that Unicode value removed. + +--- +#### psfsetversion +Usage: **`psfsetversion font [newversion]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This updates the various font version fields based on 'newversion' supplied which may be: + +- Full openTypeNameVersion string, except for the opening "Version " text, eg "1.234 beta2" +- +1 to increment to the patch version number - see below +- not supplied, in which case the current values will be validated and existing openTypeNameVersion string displayed + +It will update the openTypeNameVersion, versionMajor and versionMinor fields. It works assuming that openTypeNameVersion is of the form: + +"Version M.mpp" or "Version M.mpp extrainfo", eg "Version 1.323 Beta2" + + +Based on [FDBP](https://silnrsi.github.io/FDBP/en-US/Versioning.html), the version number is parsed as M.mpp where M is major, m is minor and pp is patch number. M matches the versionMajor and mpp the versionMinor fields. + +Incrementing will fail if either the openTypeNameVersion is not formatted correctly or the version numbers in there don’t match those in versionMajor and versionMinor. + +Examples of usage: + +``` +psfsetversion font.ufo "1.423" +``` + +will set: + +- openTypeNameVersion to "Version 1.423" +- versionMajor to 1 +- versionMinor to 423 + +`psfsetversion font.ufo +1` + +If values were originally as in the first example, openTypeNameVersion will be changed to "Version 1.424" and versionMinor to 424 + +Note that only fontinfo.plist is updated, so the font is not normalized and Pysilfont's backup mechanism for fonts is not used. + +--- +#### psfshownames +Usage: **`psfshownames [--bits] [--multiline] ifont [infont1 [ifont2]]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Show fields from the name table and optionally various bits +that control linking font styles into font families. +The default output is a table, use the `--multiline` option to produce a line oriented output. + +Fonts have bits to indicate if they are a Regular (R), Bold (B), and or Italic (I) style. +There are two groups of bits, one for Microsoft Windows (W), one for Apple macOS (M). +If a bit for a particular style is set a W and/or M is shown to indicate which platform's +set of bits is set. For Regular, there is not a corresponding bit for macOS. +Therefore, if the Regular bit is set for Windows, then W- is displayed. +The dash (-) indicates that macOS does not have that particular bit, not that the bit is 0. + +Similarly, there are bits for macOS to indicate if a font is normal, condensed, or extended width. +Windows uses a number to indicate the width. +As with the Regular style, there is no bit on macOS for normal width. + +More details are at the [Name String Example](https://docs.microsoft.com/en-us/typography/opentype/spec/namesmp) from Microsoft. + +Example usage: + +``` +psfshownames --bits Padauk-Regular.ttf +``` + +--- +#### psfsubset +Usage: **`psfsubset -i INPUT [--header HEADER] [--filter FILTER] ifont ofont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This script writes an output UFO that is a subset of the input UFO. The subset contains only the glyphs identified in the INPUT file (plus any components needed for them). + +The INPUT file can be a plain text file (one glyph per line) or a csv file. In the case of csv, the HEADER parameter is used to indicate which column from the csv to use (default is `glyph_name`). + +FILTER can be used to further reduce the subset of glyphs to only those with `Y` in the named csv column. + +Glyphs can be identified either by their name or the Unicode codepoint (USV). Glyph names and USVs can be intermixed in the list: anything that is between 4 and 6 hexadecimal digits is first processed as a USV and then, if there is no glyph encoded with that USV, processed as a glyph name. + +Glyph orders and psname mappings, if present in the font, are likewise subsetted. + +--- +#### psfsyncmasters +Usage: **`psfsyncmasters primaryds [secondds] [-n]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Synchronises/validates some fontinfo.plist and lib.plist metadata across a family of fonts based +on a designspace file. It looks in the designspace file for a master with `info copy="1"` set then syncs the values from that master to other masters defined in the file. + +If a second designspace file is supplied, it also syncs to masters found in there. + +Example usage: + +``` +psfsyncmasters CharisSIL.designspace +``` + +Note that only fontinfo.plist and lib.plist files are updated, so fonts are not normalized and Pysilfont's backup mechanism for fonts is not used. + +-n (--new) appends \_new to ufo and file names for testing purposes + +Note that currently the code also assumes a family is complex if the ufo names include 'master', though this check will +removed in the future, so `--complex` should be used instead of relying on this. + +--- +#### psfsyncmeta +Usage: **`psfsyncmeta [-s] [-m [MASTER]] [-r] [-n] [--normalize] ifont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Verifies and synchronises some fontinfo.plist and lib.plist metadata across a family of fonts. By default it uses the regular font as the master and updates any other fonts that exist assuming standard name endings of -Regular, -Italic, -Bold and -BoldItalic. Optionally a single font file can be synced against any other font as master, regardless of file naming. + +Example usage for family of fonts: + +``` +psfsyncmeta CharisSIL-Regular.ufo +``` + +This will sync the metadata in CharisSIL-Italic, CharisSIL-Bold and CharisSIL-BoldItalic against values in CharisSIL-Regular. In addition it will verify certain fields in all fonts (including Regular) are valid and follow [FDBP](https://silnrsi.github.io/FDBP/en-US/index.html) best-practice standards. + +Example usages for a single font: + +``` +psfsyncmeta -s font-Italic.ufo +psfsyncmeta -s font-Italic.ufo -m otherfont.ufo +``` +The first will sync font-Italic against font-Regular and the second against otherfont. + +Look in psfsyncmeta.py for a full details of metadata actions. + +Note that by default only fontinfo.plist and lib.plist are updated, so fonts are not normalized. Use \-\-normalize to additionally normalize all fonts in the family. + +Also psfsyncmeta does not use Pysilfont's backup mechanism for fonts. + +-n (--new) appends \_new to ufo and file names for testing purposes + +--- +#### psftuneraliases +Usage: **`psftuneraliases [-m map.csv] [-f font.ttf] feat_in.xml feat_out.xml`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Merges lookup identifiers gleaned from the map.csv file (emitted from [psfbuildfea](#psfbuildfea)), along with OpenType and Graphite feature identifiers (obtained from a compiled font), into the `<aliases>` section of a TypeTuner features.xml file. At least one of `-m` and `-f` must be provided. + +Aliases for OpenType features will generated only for the default language of each script and the alias names will be of the form `<featureTag>_<scriptTag>_dflt`. Alias names for Graphite features will be of the form `gr_<featureID>`. + +As per prior technology, the OpenType feature alias names do not distinguish between GSUB and GPOS lookups and features, therefore using the same lookup name or feature tag for both GSUB and GPOS will cause the program to exit with an error. + +--- +#### psfufo2glyphs +Usage: **`psfufo2glyphs [--glyphsformat GLYPHSFORMAT] [--nofea] designspace [glyphsfile]`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This generate a glyphs files from a designspace file and associated UFO(s). + +If no glyphsfile is specified, one will be created in the same directory as the designspace file and based on the name of the designspace file. + +By default it creates the glyphs file in glyphs v2 file format, but this can be changed using `--glyphsformat 3`. +Note that round-tripping using v3 format has not yet been tested. + +Use `--nofea` to suppress processing of any features.fea files present in the UFOs. + +Example usage: + +``` +psfglyphs2ufo AndikaItalic.designspace AndikaItalic.glyphs +``` + +Note: This is just bare-bones code at present so does the same as glyphsLib's ufo2glyphs command. It was designed so that data could be massaged, if necessary, on the way but no such need has been found so far + +--- +#### psfufo2ttf +Usage: **`psfufo2ttf [--removeOverlaps] [--decomposeComponents] iufo ottf`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +This is based on ufo2ft and generates a ttf file without OpenType tables from a UFO. + +If `--removeOverlaps` is used it merges overlapping contours + +ufo2ft filters + +- The decomposeTransformedComponents and flattenComponents filters are always used +- decomposeComponents is used if `--decomposeComponents` is set (or the filter is set in lib.plist) +- Other ufo2ft filters can also be set in lib.plist - see ufo2ft documentation for details + +--- +#### psfversion +Usage: **`psfversion`** + +This displays version info for pysilfont and many of its dependencies. It is intended for troubleshooting purposes - eg send the output in if reporting a problem - and includes which version of Python is being used and where the code is being executed from. + +--- +#### psfwoffit +Usage: **`usage: psfwoffit[-m METADATA] [--privatedata PRIVATEDATA] [-v VERSION] + [--ttf [TTF] [--woff [WOFF]] [--woff2 [WOFF2]] infont`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Converts between ttf, woff, and woff2 + +required arguments: + +``` + infont an input font; can be ttf, woff, or woff2 +``` + +optional arguments: +``` + -m METADATA, --metadata METADATA + file containing XML WOFF metadata + --privatedata PRIVATEDATA + file containing WOFF privatedata + -v VERSION, --version VERSION + woff font version number in major.minor + --ttf [TTF] name of ttf file to be written + --woff [WOFF] name of woff file to be written + --woff2 [WOFF2] name of woff2 file to be written +``` +The `--version`, `--metatadata` and `--privatedata` provide data to be added to the WOFF file. +Each of these is optional and if absent the following rules apply + +* if the input file is woff or woff2: + * the missing values are copied from the input file +* if the input file is a ttf: + * missing `metadata` or `privatedata` will be empty in the output fonts + * the version will be taken from the `fontRevison` field of the `head` table of the input ttf file. + +The output filenames can be omitted (as long as another option follows) or `-`; in either case +the output filename are calculated from the `infont` parameter. + +Examples: + +``` +psfwoffit --woff2 output.woff2 input.woff +``` +creates an output woff2 font file from an input woff font file, copying the version as well any metadata and privatedata from the input woff. + +``` +psfwoffit -m woffmetadata.xml -v 1.3 --woff output.woff --woff2 output.woff2 input.ttf +``` +creates explicitly named woff and woff2 font files from an input ttf, setting woff metadata and version from the command line. + +``` +psfwoffit --woff --woff2 -m woffmetadata.xml -v 1.3 path/font.ttf +``` +creates implicitly named `path/font.woff` and `path/font.woff2`, setting woff metadata and version from the command line. + +``` +psfwoffit --woff - --woff2 - -m woffmetadata.xml path/to/font.ttf +``` +same as above but uses font version from the ttf. + +--- +#### psfxml2compdef +Usage: **`psfxml2compdef input output`** + +_([Standard options](docs.md#standard-command-line-options) also apply)_ + +Convert composite definition file from XML format + +_This section is Work In Progress!_ + +- input Input file of CD in XML format +- output Output file of CD in single line format + + +--- + +### Specifying colors on the command line + +A color can be specified as a name (eg g_dark_green) or in RBGA format (eg (0,0.67,0.91,1)). + +Where multiple colors are supplied they should be separated by commas. + +Two special cases are also allowed (if applicable for the script): + +- "none" where the color should be removed +- "leave" where any existing color should be left unchanged + +All values are case-insensitive + +Examples: + +``` +--colors="g_cyan,g_blue" +--colors="g_dark_green, leave, (0,0.67,0.91,1)" +--color=g_red +``` +Color names can be one of the 12 cell colors definable in the GlyphsApp UI: g_red, g_orange, g_brown, g_yellow, + g_light_green, g_dark_green, g_cyan, g_blue, g_purple, g_pink, g_light_gray, g_dark_gray. + + + +## Example Scripts + +When Pysilfont is downloaded, there are example scripts in pysilfont/examples. These are a mixture of scripts that are under development, scripts that work but would likely need amending for a specific project's need and others that are just examples of how things could be done. Deprecated scripts are also placed in there. + +See [examples.md](examples.md) for further information diff --git a/docs/technical.md b/docs/technical.md new file mode 100644 index 0000000..ffbdbec --- /dev/null +++ b/docs/technical.md @@ -0,0 +1,342 @@ +# Pysilfont Technical Documentation +This section is for script writers and developers. + +See [docs.md](docs.md) for the main Pysilfont user documentation. + +# Writing scripts +The Pysilfont modules are designed so that all scripts operate using a standard framework based on the execute() command in core.py. The purpose of the framework is to: +- Simplify the writing of scripts, with much work (eg parameter parsing, opening fonts) being handled there rather than within the script. +- Provide a consistent user interface for all Pysilfont command-line scripts + +The framework covers: +- Parsing arguments (parameters and options) +- Defaults for arguments +- Extended parameter support by command-line or config file +- Producing help text +- Opening fonts and other files +- Outputting fonts (including normalization for UFO fonts) +- Initial error handling +- Reporting (logging) - both to screen and log file + +## Basic use of the framework + +The structure of a command-line script should be: +``` +<header lines> +<general imports, if any> + +from silfont.core import execute + +argspec = [ <parameter/option definitions> ] + +def doit(args): + <main script code> + return <output font, if any> + +<other function definitions> + +def cmd() : execute(Tool,doit, argspec) +if __name__ == "__main__": cmd() +``` + +The following sections work through this, using psfnormalize, which normalizes a UFO, with the option to convert between different UFO versions: +``` +#!/usr/bin/env python3 +'''Normalize a UFO and optionally convert between UFO2 and UFO3. +- If no options are chosen, the output font will simply be a normalized version of the font.''' +__url__ = 'https://github.com/silnrsi/pysilfont' +__copyright__ = 'Copyright (c) 2015 SIL International (https://www.sil.org)' +__license__ = 'Released under the MIT License (https://opensource.org/licenses/MIT)' +__author__ = 'David Raymond' + +from silfont.core import execute + +argspec = [ + ('ifont',{'help': 'Input font file'}, {'type': 'infont'}), + ('ofont',{'help': 'Output font file','nargs': '?' }, {'type': 'outfont'}), + ('-l','--log',{'help': 'Log file'}, {'type': 'outfile', 'def': '_conv.log'}), + ('-v','--version',{'help': 'UFO version to convert to'},{})] + +def doit(args) : + + if args.version is not None : args.ifont.outparams['UFOversion'] = args.version + + return args.ifont + +def cmd() : execute("UFO",doit, argspec) +if __name__ == "__main__": cmd() +``` +#### Header lines +Sample headers: +``` +#!/usr/bin/env python3 +'''Normalize a UFO and optionally convert between UFO2 and UFO3. +- If no options are chosen, the output font will simply be a normalized version of the font.''' +__url__ = 'https://github.com/silnrsi/pysilfont' +__copyright__ = 'Copyright (c) 2015 SIL International (https://www.sil.org)' +__license__ = 'Released under the MIT License (https://opensource.org/licenses/MIT)' +__author__ = 'David Raymond' +``` +As well as providing the information for someone looking at the source file, the description comment (second line, which can be multi-line) is used by the framework when constructing the help text. + +#### Import statement(s) +``` +from silfont.core import execute +``` +is required. Other imports from pysilfont or other libraries should be added, if needed. +#### Argument specification +The argument specifications take the form of a list of tuples, with one tuple per argument, eg: +``` +argspec = [ + ('ifont',{'help': 'Input font file'}, {'type': 'infont'}), + ('ofont',{'help': 'Output font file','nargs': '?' }, {'type': 'outfont'}), + ('-l','--log',{'help': 'Log file'}, {'type': 'outfile', 'def': '_conv.log'}), + ('-v','--version',{'help': 'UFO version to convert to'},{})] +``` +Each argument has the format: +``` +(argument name(s),argparse dict,framework dict) +``` +argument name is either +- name for positional parameters, eg *‘ifont’* +- *-n, --name* or *--name* for other arguments, eg *‘-v’, ‘--version’* + +**argparse dict** follows standard [argparse usage for .add_argument()](https://docs.python.org/2/library/argparse.html#the-add-argument-method). Help should always be included. + +**framework dict** has optional values for: +- ‘type’ - the type of parameter, eg ‘outfile’ +- ‘def’ - default for file names. Only applies if ‘type’ is a font or file. +- 'optlog' - For logs only. Flag to indicate the log file is optional - default False + +‘Type’ can be one of: + +| Value | Action | +|-------|-------------------------------------| +|infont|Open a font of that name and pass the font to the main function| +|outfont|If the main function to returns a font, save that to the supplied name| +|infile|Open a file for read and pass the file handle to the main function| +|incsv|Open a [csv](#support-for-csv-files) file for input and pass iterator to the main function| +|outfile|Open a file for writing and pass the file handle to the main function| +|filename|Filename to be passed as text| +|optiondict|Expects multiple values in the form name=val and passes a dictionary containing them| + +If ‘def’ is supplied, the parameter value is passed through the [file name defaulting](#default-values-for-arguments) as specified below. Applies to all the above types except for optiondict. + +In addition to options supplied in argspec, the framework adds [standard options](docs.md#standard-command-line-options), ie: + +- -h, --help +- -q, --quiet +- -p, --params +- -l, --log + +so these do not need to be included in argspec. + +With -l, --log, this is still usually set in argspec to create default log file names. Set optlog to False if you want the log file to be optional. + +#### doit() function +The main code of the script is in the doit() function. + +The name is just by convention - it just needs to match what is passed to execute() at the end of the script. The +execute() function passes an args object to doit() containing: +- An entry for each command-line argument as appropriate, based on the full name of the argument + - eg with ``'-v','--version'``, args.version is set. + - Values are set for every entry in argspec, plus params, quiet and log added by the framework + - If no value is given on the command-line and the argument has no default then None is used. +- logger for the loggerobj() +- clarguments for a list of what was actually specified on the command line +- For parameters: + - params is a list of what parameters, if any, were specified on the command line + - paramsobj is the parameters object containing all [parameter](parameters.md) details + +#### The final lines + +These should always be: +``` +def cmd() : execute(Tool,doit, argspec) +if __name__ == "__main__": cmd() +``` +The first line defines the function that actually calls execute() to do the work, where Tool is one of: +- “UFO” to open fonts with pysilfont’s ufo.py module, returning a Ufont object +- “FP” to open fonts with fontParts, returning a font object +- “FT” to open fonts with FontTools, returning a TTfont object +- None if no font to be opened by execute() +- Other tools may be added in the future + +The function must be called cmd(), since this is used by setup.py to install the commands. + +The second line is the python way of saying, if you run this file as a script (rather than using it as a python module), execute the cmd() function. + +Even if a script is initially just going to be used to be run manually, include these lines so no modification is needed to make it installable at a later date. + +# Further framework notes +## Default values for arguments +Default values in [docs.md](docs.md#default-values) describes how file name defaulting works from a user perspective. + +To set default values, either use the ‘default’ keyword in the argparse dict (for standard defaults) or the ‘def’ keyword in the framework dict to use Pysilfont’s file-name defaulting mechanism. Only one of these should be used. 'def' can't be used with the first positional parameter. + +Note if you want a fixed file name, ie to bypass the file name defaulting mechanism, then use the argparse default keyword. + +## Reporting +args.logger is a loggerobj(), and used to report messages to screen and log file. If no log file is set, messages are just to screen. + +Messages are sent using +``` +logger.log(<message text>, [severity level]> +``` +Where severity level has a default value of W and can be set to one of: +- X Exception - For fatal programming errors +- S Severe - For fatal errors - eg input file missing +- E Errors - For serious errors that must be reported to screen +- P Progress - Progress messages +- W Warning - General warnings about anything not correct +- I Info - For more detailed reporting - eg the name of each glif file opened +- V Verbose - For even more messages! + +Errors are reported to screen if the severity level is higher or equal to logger.scrlevel (default P) and to log based on loglevel (default W). The defaults for these can be set via parameters or within a script, if needed. + +With X and S, the script is terminated. S should be used for user problems (eg file does not exist, font is invalid) and X for programming issues (eg an invalid value has been set by code). Exception errors are mainly used by the libraries and force a stack trace. + +With Ufont objects, font.logger also points to the logger, but this is used primarily within the libraries rather than in scripts. + +There would normally only be a single logger object used by a script. + +### Changing reporting levels + +loglevel and scrlevel *can* be set by scripts, but care should be taken not to override values set on the command line. To increase screen logging temporarily, use logger.raisescrlevel(<new level>) then set to previous value with logger.resetscrlevel(), eg + +``` + if not(args.quiet or "scrlevel" in params.sets["command line"]) : + logger.raisescrlevel("W") # Raise level to W if not already W or higher + + <code> + + logger.resetscrlevel() +``` + +### Error and warning counts + +These are kept in logger.errorcount and logger.warningcount. + +For scripts using the execute() framework, these counts are reported as progress messages when the script completes. + +## Support for csv files +csv file support has been added to core.py with a csvreader() object (using the python csv module). In addition to the basic handling that the csv module provides, the following are supported: +- csvreader.firstline returns the first line of the file, so analyse headers if needed. Iteration still starts with the first line. +- Specifying the number of values expected (with minfields, maxfields, numfields) +- Comments (lines starting with #) are ignored +- Blank lines are also ignored + +The csvreader() object is an iterator which returns the next line in the file after validating it against the min, max and num settings, if any, so the script does not have to do such validation. For example: +``` +incsv = csvreader(<filespec>) +incsv.minfields = 2 +Incsv.maxfields = 3 +for line in inscv: + <code> +``` +Will run `<code>` against each line in the file, skipping comments and blank lines. If any lines don’t have 2 or 3 fields, an error will be reported and the line skipped. + +## Parameters +[Parameters.md](parameters.md) contains user, technical and developer’s notes on these. + +## Chaining +With ufo.py scripts, core.py has a mechanism for chaining script function calls together to avoid writing a font to disk then reading it in again for the next call. In theory it could be used simply to call another script’s function from within a script. + +This has not yet been used in practice, and will be documented (and perhaps debugged!) when there is a need, but there are example scripts to show how it was designed to work. + +# pysilfont modules + +These notes should be read in conjunction with looking at the comments in the code (and the code itself!). + +## core.py + +This is the main module that has the code to support: +- Reporting +- Logging +- The execute() function +- Chaining +- csvreader() + +## etutil.py + +Code to support xml handling based on xml.etree cElementTree objects. It covers the following: +- ETWriter() - a general purpose pretty-printer for outputting xml in a normalized form including + - Various controls on indenting + - inline elements + - Sorting attributes based on a supplied order + - Setting decimal precision for specific attributes + - doctype, comments and commentsafter +- xmlitem() class + - For reading and writing xml files + - Keeps record of original and final xml strings, so only needs to write to disk if changed +- ETelement() class + - For handling an ElementTree element + - For each tag in the element, ETelement[tag] returns a list of sub-elements with that tag + - process_attributes() processes the attributes of the element based on a supplied spec + - process_subelements() processes the subelements of the element based on a supplied spec + +xmlitem() and ETelement() are mainly used as parent classes for other classes, eg in ufo.py. + +The process functions validate the attributes/subelements against the spec. See code comments for details. + +#### Immutable containers + +Both xmlitem and ETelement objects are immutable containers, where +- object[name] can be used to reference items +- the object can be iterated over +- object.keys() returns a list of keys in the object + +however, values can't be set with `object[name] = ... `; rather values need to be set using methods within child objects. For example, with a Uglif object, you can refer to the Uadvance object with glif['advance'], but to add a Uadvance object you need to use glif.addObject(). + +This is done so that values can be easily referenced and iterated over, but values can only be changed if appropriate methods have been defined. + +Other Pysilfont objects also use such immutable containers. + +## util.py + +Module for general utilities. Currently just contains dirtree code. + +#### dirTree + +A dirTree() object represents all the directories and files in a directory tree and keeps track of the status of the directories/files in various ways. It was designed for use with ufo.py, so, after changes to the ufo, only files that had been added or changed were written to disk and files that were no longer part of the ufo were deleted. Could have other uses! + +Each dirTreeItem() in the tree has details about the directory or file: +- type + - "d" or "f" to indicate directory or file +- dirtree + - For sub-directories, a dirtree() for the sub-directory +- read + - Item has been read by the script +- added + - Item has been added to dirtree, so does not exist on disk +- changed + - Item has been changed, so may need updating on disk +- towrite + - Item should be written out to disk +- written + - Item has been written to disk +- fileObject + - An object representing the file +- fileType + - The type of the file object +- flags + - Any other flags a script might need + + +## ufo.py + +See [ufo.md](ufo.md) for details + +## ftml.py + +To be written + +## comp.py + +To be written + +# Developer's notes + +To cover items relevant to extending the library modules or adding new + +To be written diff --git a/docs/tests.md b/docs/tests.md new file mode 100644 index 0000000..6007470 --- /dev/null +++ b/docs/tests.md @@ -0,0 +1,17 @@ +# Test suite +pysilfont has a pytest-based test suite. + +### install the test framework bits: +``` +python3 -m pip install pytest +``` + +## set up the folders: +``` +python3 tests/setuptestdata.py +``` + +## run the test suite: +``` +pytest +``` diff --git a/docs/ufo.md b/docs/ufo.md new file mode 100644 index 0000000..55049d7 --- /dev/null +++ b/docs/ufo.md @@ -0,0 +1,159 @@ +# Pysilfont - ufo support technical docs + +# The Basics + +UFO support is provided by the ufo.py library. + +Most scripts work by reading a UFO into a Ufont object, making changes to it and writing it out again. The Ufont object contains many separate objects representing the UFO in a UFO 3 based hierarchy, even if the original UFO was format 2 - see [UFO 2 or UFO 3?](#ufo-2-or-ufo-3-) below. + +Details of the [Ufont Object Model](#ufont-object-model) are given below, but in summary: + +- There is an object for each file within the UFO (based on [xmlitem](technical.md#etutil.py)) +- There is an object for each xml element within a parent object (based on [ETelement](technical.md#etutil.py)) +- Data within objects can always(?) be accessed via generic methods based on the xml element tags +- For much data, there are object-specific methods to access data, which is easier than the generic methods + +For example, a .glif file is: +- Read into a Uglif object which has: + - Methods for glyph-level data (eg name, format) + - objects for all the sub-elements within a glyph (eg advance, outline) + - Where an element type can only appear once in a glyph, eg advance, Uglif.*element-name* returns the relevant object + - Where an element can occur multiple times (eg anchor), Uglif.*element-name* returns a list of objects +- If an sub-element itself has sub-elements, then there are usually sub-element objects for that following the same pattern, eg Uglif.outline has lists of Ucontour and Ucomponent objects + +It is planned that more object-specific methods will be added as needs arise, so raise in issue if you see specific needs that are likely to be useful in multiple scripts. + + + +### UFO 2 or UFO 3? + +The Ufont() object model UFO 3 based, so UFO 2 format fonts are converted to UFO 3 when read and then converted back to UFO 2 when written out to disk. Unless a script has code that is specific to a particular UFO format, scripts do not need to know the format of the font that was opened; rather they can just work in the UFO 3 format and leave the conversion to the library. + +The main differences this makes in practice are: +- **Layers** The Ufont() always has layers. With UFO 2 fonts, there will be only one, and it can be accessed via Ufont.deflayer +- **Anchors** If a UFO 2 font uses the accepted practice of anchors being single point contours with a name and a type of "Move" then + - On reading the font, they will be removed from the list of contours and added to the list of anchors + - On writing the font, they will be added back into the list of contours + +Despite being based on UFO 3 (for future-proofing), nearly all use of Pysilfont's UFO scripts has been with UFO 2-based projects so testing with UFO 3 has been minimal - and there are some [known limitations](docs.md#known-limitations). + + +# Ufont Object Model + +A **Ufont** object represents the font using the following objects: + +- font.**metainfo**: [Uplist](#uplist) object created from metainfo.plist +- font.**fontinfo**: [Uplist](#uplist) object created from fontinfo.plist, if present +- font.**groups**: [Uplist](#uplist) object created from groups.plist, if present +- font.**kerning**: [Uplist](#uplist) object created from kerning.plist, if present +- font.**lib**: [Uplist](#uplist) object created from lib.plist, if present +- self.**layercontents**: [Uplist](#uplist) object + - created from layercontents.plist for UFO 3 fonts + - synthesized based on the font's single layer for UFO 2 fonts +- font.**layers**: List of [Ulayer](#ulayer) objects, where each layer contains: + - layer.**contents**: [Uplist](#uplist) object created from contents.plist + - layer[**_glyph name_**]: A [Uglif](#uglif) object for each glyph in the font which contains: + - glif['**advance**']: Uadvance object + - glif['**unicode**']: List of Uunicode objects + - glif['**note**']: Unote object (UFO 3 only) + - glif['**image**']: Uimage object (UFO 3 only) + - glif['**guideline**']: List of Uguideline objects (UFO 3 only) + - glif['**anchor**']: List of Uanchor objects + - glif['**outline**']: Uoutline object which contains + - outline.**contours**: List of Ucontour objects + - outline.**components**: List of Ucomponent objects + - glif['**lib**']: Ulib object +- font.**features**: UfeatureFile created from features.fea, if present + +## General Notes + +Except for UfeatureFile (and Ufont itself), all the above objects are set up as [immutable containers](technical.md#immutable-containers), though the contents, if any, depend on the particular object. + +Objects usually have a link back to their parent object, eg glif.layer points to the Ulayer object containing that glif. + +## Specific classes + +**Note - the sections below don't list all the class details** so also look in the code in ufo.py if you need something not listed - it might be there! + +### Ufont + +In addition to the objects listed above, a Ufont object contains: +- self.params: The [parameters](parameters.md) object for the font +- self.paramsset: The parameter set within self.params specific to the font +- self.logger: The logger object for the script +- self.ufodir: Text string of the UFO location +- self.UFOversion: from formatVersion in metainfo.plist +- self.dtree: [dirTree](technical.md#dirtree) object representing all the files on fisk and their status +- self.outparams: The output parameters for the font, initially set from self.paramset +- self.deflayer: + - The single layer for UFO 2 fonts + - The layer called public.default for UFO 3 fonts + +When creating a new Ufont() object in a script, it is normal to pass args.paramsobj for params so that it has all the settings for parameters and logging. + +self.write(outputdir) will write the UFO to disk. For basic scripts this will usually be done by the execute() function - see [writing scripts](technical.md#writing-scripts). + +self.addfile(type) will add an empty entry for any of the optional plist files (fontinfo, groups, kerning or lib). + +When writing to disk, the UFO is always normalized, and only changed files will actually be written to disk. The format for normalization, as well as the output UFO version, are controlled by values in self.outparams. + +### Uplist + +Used to represent any .plist file, as listed above. + +For each key,value pair in the file, self[key] contains a list: +- self[key][0] is an elementtree element for the key +- self[key][1] is an elementtree element for the value + +self.getval(key) will return: +- the value, if the value type is integer, real or string +- a list if the value type is array +- a dict if the value type is dict +- None, for other value types (which would only occur in lib.plist) +- It will throw an exception if the key does not exist +- for dict and array, it will recursively process dict and/or array subelements + +Methods are available for adding, changing and deleting values - see class \_plist in ufo.py for details. + +self.font points to the parent Ufont object + +### Ulayer + +Represents a layer in the font. With UFO 2 fonts, a single layer is synthesized from the glifs folder. + +For each glyph, layer[glyphname] returns a Uglif object for the glyph. It has addGlyph and delGlyph functions. + +### Uglif + +Represents a glyph within a layer. It has child objects, as listed below, and functions self.add and self.remove for adding and removing them. For UFO 2 fonts, and contours identified as anchors will have been removed from Uoutline and added as Uanchor objects. + +#### glif child objects + +There are 8 child objects for a glif: + +| Name | Notes | UFO 2 | Multi | +| ---- | -------------------------------- | --- | --- | +| Uadvance | Has width & height attributes | Y | | +| Uunicode | Has hex attribute | Y | Y | +| Uoutline | | Y | | +| Ulib | | Y | | +| Unote | | | | +| Uimage | | | | +| Uguideline | | | Y | +| Uanchor | | | Y | + +They all have separate object classes, but currently (except for setting attributes), only Uoutline and Ulib have any extra code - though more will be added in the future. + +(With **Uanchor**, the conversion between UFO 3 anchors and the UFO 2 way of handling anchors is handled by code in Uglif and Ucontour) + +**Ulib** shares a parent class (\_plist) with [Uplist](#uplist) so has the same functionality for managing key,value pairs. + +#### Uoutline + +This has Ucomponent and Ucontour child objects, with addobject, appendobject and insertobject methods for managing them. + +With Ucontour, self['point'] returns a list of the point subelements within the contour, and points can be managed using the methods in Ulelement. other than that, changes need to be made by changing the elements using elementtree methods. + +# Module Developer Notes + +To be written |