Many More Letters
News | Rob Meek (meek) | July 9th, 2021
Dear FontStructors,
Today we’re adding many more letters (the entire Unicode 13 standard) to FontStruct.
For those of you unfamiliar with the term, Unicode is a huge long list of all the letters and letter-like things which are used by all kinds of writing systems from all parts of the world. This comprehensive list is maintained by an international group of experts. Every letter (or letter-like thing) is assigned a code number, and these numbers are used to help identify letters inside computer fonts.
Before today, FontStruct supported only parts of Unicode, but now we’re finally providing access to all of the “code blocks” (i.e. all the letters) defined in the most recently released standard (Unicode 13). Now you could design a FontStruction containing as many as 143,859 letters*! I wouldn’t recommend that though. Once you get over about 10,000, you will find that the site and editor will start struggling.
The main reason for expanding the number of letters available on FontStruct is to allow designers to access and design a greater variety of scripts, in particular writing systems – such as Chinese scripts – which use a large number of characters, and also scripts, such as Sogdian or Old Sogdian for example, which we’ve simply overlooked until now. You can also now access the Emoji slots.
There’s still a long way to go in order to fully support the variety of scripts we’d like to see on FontStruct, but supporting Unicode is, I hope, a good start.
You will find all the new letters by selecting “Expert Mode” and then “Menu”, “Advanced”, “Unicode Letter Sets”:
The letter-set select box in the bottom-left corner of the FontStructor has changed. There are many more options of course (360 letter sets), and also a handy new search box to quickly find what you are looking for (desktop only). Note that we broke up some of the huge code blocks, such as the CJK extensions, into smaller, numbered blocks.
Full unicode support has been a very frequent request over the years, so it’s great to finally be able to deliver.
Thanks go, as ever, to our Patrons, and to our sponsors GlyphsApp (the world’s leading desktop type-design software) whose continuing support makes the development of features like this possible.
Happy FontStructing!
(PS: We still don’t provide access to codepoints below 30 – control characters.)
*As @BWM pointed out in the comments, there is a technical limit for TrueType fonts of 65,535 characters, so there’s certainly no point in going beyond that.
WOW! Very extended!
– elmoyenique — July 9, 2021 #
It looks like fontstructions fail to save when glyphs outside the BMP (U+0000 to U+FFFF) are present. I’ve heard of this happening before, so maybe it just wasn’t fixed? Other than that, it’s nice to be able to access the Private Use Area. (Or at least the one in the BMP.)
– realicraft — July 9, 2021 #
Thanks @realicraft I will look into that.
(EDIT: I’ve identified the problem. It’s simple, but will take a while to fix).
– Rob Meek (meek) — July 9, 2021 #
Wow!
– MetrikEnzyme — July 9, 2021 #
@Meek Thanks for this, Now I should be able to create fonts for certain scripts without needing to use FontForge to move the letters into the correct codepoints…
I noticed that using the <- and -> arrows to move between letters does not work some of the time…
Plus, the glyph limit is a hard 65,535 for fonts (including non-unicode codepoints like Small caps, Devanagari consonant clusters, old style numerals, the .notdef, etc.), so if you do need more glyphs than that, you can create more than one font and link them together in the description. (For example, if your font is named GeneaClock, you can create another font named GeneaClock-Suppl1 and place a link to the original font in the description)
I’ll also edit my Minecraftio font to include the gothic letters when it does save letters outside the BMP.
– BWM — July 9, 2021 #
@BWM Yes I noticed the issue with arrows as well, and I wasn’t aware of the hard limit on glyph counts. I’ll update the blog post to reflect that.
Also, just to confirm @realicraft’s point, for now only codepoints up to U+FFFF can be saved. It’s proving a little harder than I’d hoped to fix that. Could be a few days or a week.
– Rob Meek (meek) — July 9, 2021 #
First layered color fonts, and now this! FS is truly a gift that keeps on giving! Honestly, I was so overwhelmed with the concept of layers and colors being introduced that I could neither talk about them nor use them until now. Thanks for the continuous expansion of innovation and functionality you bring to us, meek!
– geneus1 — July 9, 2021 #
I found a way to encode SMP characters!
– MetrikEnzyme — July 9, 2021 #
I’ve been waiting a long time for this! (Would’ve been longer if I never learned how to hack the browser, but hey.)
@MetrikEnzyme How?
– erictom333 — July 9, 2021 #
@erictom333
You could encode Ԁ to ? because they have the same four last digits!
– MetrikEnzyme — July 10, 2021 #
You should be able to save codepoints beyond U+FFFF now.
– Rob Meek (meek) — July 10, 2021 #
@MetrikEnzyme I thought of that earlier, but decided against because 1) I wanted them in the same font and 2) U+10000 to U+1001F (part of Linear B) could not be encoded as U+0000 to U+001F are control characters.
I’ve just developed a font (SMP Test) that has the Old Italic and Gothic scripts. However, the characters do not display in the preview, and when I download it, the characters, which would normally be over 10300-1034F, are in 0300-034F in Character Map. (Character Map does not display SMP characters.)
– erictom333 — July 10, 2021 #
So just encode 0300-034F!
– MetrikEnzyme — July 10, 2021 #
See reason 1. (SMP Test is just a test, Pixel Sans is the real deal.)
– erictom333 — July 10, 2021 #
Another recommendation: The editor lags unbearably when editing blocks of 4096 (like CJK). Could someone split them into smaller blocks of 256 or 512?
– erictom333 — July 10, 2021 #
Also, I added two extended kana (1B000 and 1B001) to Pixel Sans and now when I download and install it, it says that it is not a valid font file.
– erictom333 — July 10, 2021 #
Maybe you installed the .otf file?
– MetrikEnzyme — July 10, 2021 #
There is an issue with the font file generation (download). It’s being worked on.
– Rob Meek (meek) — July 10, 2021 #
@MetrikEnzyme No, I trid both files and neither worked.
Another bug: The previous and next character shortcuts sometimes do not work in the SMP.
– erictom333 — July 10, 2021 #
Suggestion: Split CJK by frequency in Chinese/Japanese, or allow users to load their own custom lists (draw arbitrary characters instead of hunting).
– Umbreon126 — July 10, 2021 #
This is amazing news! Cannot wait to try this out
– CMunk — July 12, 2021 #
Continuing on @Umbreon126’s suggestion:
I have a proposal that would solve the problem of having to scrollthrough thousands of CJK, Hangul or PUA characters. The Unicode Character Set switchshould be moved from the menu to next to the exisging charsacter subset selector, and become a dropdown menu, with the folloeing options:
– Basic: As present.
– Unicode: As present. Large (>700 chars) blocks would be split into 256- or 512-character blocks.
– Legacy: Various legacy character sets, including Windows snd DOS codepages and PETSCII.
– Shift-JIS: Self-explanatory. Arranged by codepsge.
– CJK by Radical: CJK ideographs arranged by radical.
– CJK by Pinyin/Bopomofo/Hangul/Kana: Somewhat self-explanatory.
– CJK by Frequency, per @Umbreon126’s suggestion
– Hangul: Hangul syllables arranged by initial consonant.
– Not sure how Yi could be implemented
– Private Use: Arranged by 256-character block.
– Should you include support for other planesof Unicode, I’ve got similsr ideas for Cuneiform, Egyptian Hieroglyphs, Tangut, emojis, and other large scripts.
– Unicode by Code: Last resort. All of Unicode by 256-character block.
– erictom333 — July 12, 2021 #
Mr. Meek? There is this bug where if you press the left or right arrow keys, it doesn’t go to the previous or next letter sometimes. Can you fix it please?
– DarkGuy572 — July 12, 2021 #
It’s amazing, it’s beautiful, I’ve been wanting to create a font with all CJK hieroglyphs for a long time, thanQ so much! And I have an idea how to simplify navigation through scripts and thieves (signature removers):
– fontlanger2005 — July 12, 2021 #
The download issue, and the issue with the left and right arrow keys should be fixed now.
– Rob Meek (meek) — July 13, 2021 #
It’s amazing, it’s beautiful, I’ve been wanting to create a font with all CJK hieroglyphs for a long time, thanQ so much!
– echoly89699 — July 14, 2021 #
I’d like to propose a mass copy feature, which would let us copy a range of glyphs from one location to another. I need this as I have copies of some SMP glyphs in the PUA.
– erictom333 — July 14, 2021 #
I too am experiencing extreme lag in the larger blocks; perhaps they could be split further, into sets of 256 or 512? This would, in my opinion, also make the larger blocks easier to navigate.
Technical details: ASUS X555QA with AMD A12-9700P RADEON R7 4C+6G (4) @ 2.500GHz and AMD ATI Radeon R5/R6/R7 Graphics, Ubuntu 20.04.2 LTS, Vivaldi 4.0.2312.38 with hardware-accelerated graphics
– ImmaPooh — July 14, 2021 #
Thanks for the technical detail @ImmaPooh. I will take a look into that. It should be possible to optimize the current solution rather than changing the size of the split blocks, and there will be refinements to come, but probably not until after the Summer (northern hemisphere).
– Rob Meek (meek) — July 14, 2021 #