v3.12.0

Disable reflow when winptyCompat is on

CONTRIBUTING.md

@@ -33,21 +33,21 @@ opening an issue, read these pointers.
 You can find issues to work on by looking at the [help wanted](https://github.com/xtermjs/xterm.js/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) or [good first issue](https://github.com/xtermjs/xterm.js/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) issues. It's a good idea to comment on the issue saying that you're taking it, just in case someone else comes along and you duplicate work. Once you have your issue, here are the steps to contribute:
 
 - Fork [xterm.js](https://github.com/sourcelair/xterm.js/)
-  ([how to fork a repo](https://help.github.com/articles/fork-a-repo))
-- Get the [xterm.js demo](https://github.com/xtermjs/xterm.js/wiki/Contributing#running-the-demo) running
-- Make your changes
+  ([how to fork a repo](https://help.github.com/articles/fork-a-repo)).
+- Get the [xterm.js demo](https://github.com/xtermjs/xterm.js/wiki/Contributing#running-the-demo) running.
+- Make your changes.
 - If your changes are easy to test or likely to regress, add tests. Tests go into `test`, directory.
 - Follow the general code style of the rest of the project (see below).
 - Submit a pull request
 ([how to create a pull request](https://help.github.com/articles/fork-a-repo)).
   Don't put more than one feature/fix in a single pull request.
 
-By contributing code to xterm.js you
+By contributing code to xterm.js you:
 
- - agree to license the contributed code under xterm.js' [MIT
+ - Agree to license the contributed code under xterm.js' [MIT
    license](LICENSE).
 
- - confirm that you have the right to contribute and license the code
+ - Confirm that you have the right to contribute and license the code
    in question. (Either you hold all rights on the code, or the rights
    holder has explicitly granted the right to use it like this,
    through a compatible open source license or through a direct

LICENSE

@@ -1,4 +1,4 @@
-Copyright (c) 2017-2018, The xterm.js authors (https://github.com/xtermjs/xterm.js)
+Copyright (c) 2017-2019, The xterm.js authors (https://github.com/xtermjs/xterm.js)
 Copyright (c) 2014-2016, SourceLair Private Company (https://www.sourcelair.com)
 Copyright (c) 2012-2013, Christopher Jeffrey (https://github.com/chjj/)
 

README.md

@@ -7,17 +7,17 @@ Xterm.js is a front-end component written in TypeScript that lets applications b
 
 ## Features
 
-- **Terminal apps just work**: Xterm.js works with most terminal apps such as `bash`, `vim` and `tmux`, this includes support for curses-based apps and mouse event support
-- **Perfomant**: Xterm.js is *really* fast, it even includes a GPU-accelerated renderer
-- **Rich unicode support**: Supports CJK, emojis and IMEs
-- **Self-contained**: Requires zero dependencies to work
-- **Accessible**: Screen reader support can be turned on using the `screenReaderMode` option
+- **Terminal apps just work**: Xterm.js works with most terminal apps such as `bash`, `vim` and `tmux`, this includes support for curses-based apps and mouse event support.
+- **Perfomant**: Xterm.js is *really* fast, it even includes a GPU-accelerated renderer.
+- **Rich unicode support**: Supports CJK, emojis and IMEs.
+- **Self-contained**: Requires zero dependencies to work.
+- **Accessible**: Screen reader support can be turned on using the `screenReaderMode` option.
 - **And much more**: Links, theming, addons, well documented API, etc.
 
 ## What xterm.js is not
 
-- Xterm.js is not a terminal application that you can download and use on your computer
-- Xterm.js is not `bash`. Xterm.js can be connected to processes like `bash` and let you interact with them (provide input, receive output)
+- Xterm.js is not a terminal application that you can download and use on your computer.
+- Xterm.js is not `bash`. Xterm.js can be connected to processes like `bash` and let you interact with them (provide input, receive output).
 
 ## Getting Started
 
@@ -41,7 +41,7 @@ To start using xterm.js on your browser, add the `xterm.js` and `xterm.css` to t
       <script>
         var term = new Terminal();
         term.open(document.getElementById('terminal'));
-        term.write('Hello from \x1B[1;3;31mxterm.js\x1B[0m $ ')
+        term.write('Hello from \x1B[1;3;31mxterm.js\x1B[0m $ ')
       </script>
     </body>
   </html>
@@ -106,17 +106,17 @@ Note that some APIs are marked *experimental*, these are added to enable experim
 ## Real-world uses
 Xterm.js is used in several world-class applications to provide great terminal experiences.
 
-- [**SourceLair**](https://www.sourcelair.com/): In-browser IDE that provides its users with fully-featured Linux terminals based on xterm.js
-- [**Microsoft Visual Studio Code**](http://code.visualstudio.com/): Modern, versatile and powerful open source code editor that provides an integrated terminal based on xterm.js
-- [**ttyd**](https://github.com/tsl0922/ttyd): A command-line tool for sharing terminal over the web, with fully-featured terminal emulation based on xterm.js
+- [**SourceLair**](https://www.sourcelair.com/): In-browser IDE that provides its users with fully-featured Linux terminals based on xterm.js.
+- [**Microsoft Visual Studio Code**](http://code.visualstudio.com/): Modern, versatile and powerful open source code editor that provides an integrated terminal based on xterm.js.
+- [**ttyd**](https://github.com/tsl0922/ttyd): A command-line tool for sharing terminal over the web, with fully-featured terminal emulation based on xterm.js.
 - [**Katacoda**](https://www.katacoda.com/): Katacoda is an Interactive Learning Platform for software developers, covering the latest Cloud Native technologies.
 - [**Eclipse Che**](http://www.eclipse.org/che): Developer workspace server, cloud IDE, and Eclipse next-generation IDE.
 - [**Codenvy**](http://www.codenvy.com): Cloud workspaces for development teams.
-- [**CoderPad**](https://coderpad.io): Online interviewing platform for programmers. Run code in many programming languages, with results displayed by `xterm.js`.
-- [**WebSSH2**](https://github.com/billchurch/WebSSH2): A web based SSH2 client using `xterm.js`, socket.io, and ssh2.
+- [**CoderPad**](https://coderpad.io): Online interviewing platform for programmers. Run code in many programming languages, with results displayed by xterm.js.
+- [**WebSSH2**](https://github.com/billchurch/WebSSH2): A web based SSH2 client using xterm.js, socket.io, and ssh2.
 - [**Spyder Terminal**](https://github.com/spyder-ide/spyder-terminal): A full fledged system terminal embedded on Spyder IDE.
 - [**Cloud Commander**](https://cloudcmd.io "Cloud Commander"): Orthodox web file manager with console and editor.
-- [**Codevolve**](https://www.codevolve.com "Codevolve"): Online platform for interactive coding and web development courses. Live container-backed terminal uses `xterm.js`.
+- [**Codevolve**](https://www.codevolve.com "Codevolve"): Online platform for interactive coding and web development courses. Live container-backed terminal uses xterm.js.
 - [**RStudio**](https://www.rstudio.com/products/RStudio "RStudio"): RStudio is an integrated development environment (IDE) for R.
 - [**Terminal for Atom**](https://github.com/jsmecham/atom-terminal-tab): A simple terminal for the Atom text editor.
 - [**Eclipse Orion**](https://orionhub.org): A modern, open source software development environment that runs in the cloud. Code, deploy and run in the cloud.
@@ -124,9 +124,8 @@ Xterm.js is used in several world-class applications to provide great terminal e
 - [**Hexlet**](https://en.hexlet.io): Practical programming courses (JavaScript, PHP, Unix, databases, functional programming). A steady path from the first line of code to the first job.
 - [**Selenoid UI**](https://github.com/aerokube/selenoid-ui): Simple UI for the scallable golang implementation of Selenium Hub named Selenoid. We use XTerm for streaming logs over websockets from docker containers.
 - [**Portainer**](https://portainer.io): Simple management UI for Docker.
-- [**SSHy**](https://github.com/stuicey/SSHy): HTML5 Based SSHv2 Web Client with E2E encryption utilising `xterm.js`, SJCL & websockets.
-- [**JupyterLab**](https://github.com/jupyterlab/jupyterlab): An extensible
-computational environment for Jupyter, supporting interactive data science and scientific computing across all programming languages.
+- [**SSHy**](https://github.com/stuicey/SSHy): HTML5 Based SSHv2 Web Client with E2E encryption utilising xterm.js, SJCL & websockets.
+- [**JupyterLab**](https://github.com/jupyterlab/jupyterlab): An extensible computational environment for Jupyter, supporting interactive data science and scientific computing across all programming languages.
 - [**Theia**](https://github.com/theia-ide/theia): Theia is a cloud & desktop IDE framework implemented in TypeScript.
 - [**Opshell**](https://github.com/ricktbaker/opshell) Ops Helper tool to make life easier working with AWS instances across multiple organizations.
 - [**Proxmox VE**](https://www.proxmox.com/en/proxmox-ve): Proxmox VE is a complete open-source platform for enterprise virtualization. It uses xterm.js for container terminals and the host shell.
@@ -138,22 +137,23 @@ computational environment for Jupyter, supporting interactive data science and s
 - [**Azure Cloud Shell**](https://shell.azure.com): Azure Cloud Shell is a Microsoft-managed admin machine built on Azure, for Azure.
 - [**atom-xterm**](https://atom.io/packages/atom-xterm): Atom plugin for providing terminals inside your Atom workspace.
 - [**rtty**](https://github.com/zhaojh329/rtty): A reverse proxy WebTTY. It is composed of the client and the server.
-- [**Pisth**](https://github.com/ColdGrub1384/Pisth): An SFTP and SSH client for iOS
+- [**Pisth**](https://github.com/ColdGrub1384/Pisth): An SFTP and SSH client for iOS.
 - [**abstruse**](https://github.com/bleenco/abstruse): Abstruse CI is a continuous integration platform based on Node.JS and Docker.
 - [**Azure Data Studio**](https://github.com/Microsoft/azuredatastudio): A data management tool that enables working with SQL Server, Azure SQL DB and SQL DW from Windows, macOS and Linux.
-- [**FreeMAN**](https://github.com/matthew-matvei/freeman): A free, cross-platform file manager for power users
+- [**FreeMAN**](https://github.com/matthew-matvei/freeman): A free, cross-platform file manager for power users.
 - [**Fluent Terminal**](https://github.com/felixse/FluentTerminal): A terminal emulator based on UWP and web technologies.
-- [**Hyper**](https://hyper.is): A terminal built on web technologies
+- [**Hyper**](https://hyper.is): A terminal built on web technologies.
 - [**Diag**](https://diag.ai): A better way to troubleshoot problems faster. Capture, share and reapply troubleshooting knowledge so you can focus on solving problems that matter.
 - [**GoTTY**](https://github.com/yudai/gotty): A simple command line tool that shares your terminal as a web application based on xterm.js.
 - [**genact**](https://github.com/svenstaro/genact): A nonsense activity generator.
 - [**cPanel & WHM**](https://cpanel.com): The hosting platform of choice.
-- [**Nutanix**](https://github.com/nutanix): Nutanix Enterprise Cloud uses xterm in the webssh functionality within Nutanix Calm, and is also looking to move our old noserial (termjs) functionality to xterm.js
+- [**Nutanix**](https://github.com/nutanix): Nutanix Enterprise Cloud uses xterm in the webssh functionality within Nutanix Calm, and is also looking to move our old noserial (termjs) functionality to xterm.js.
 - [**SSH Web Client**](https://github.com/roke22/PHP-SSH2-Web-Client): SSH Web Client with PHP.
 - [**Shellvault**](https://www.shellvault.io): The cloud-based SSH terminal you can access from anywhere.
 - [**Juno**](http://junolab.org/): A flexible Julia IDE, based on Atom.
 - [**webssh**](https://github.com/huashengdun/webssh): Web based ssh client.
-- [**info-beamer hosted**](https://info-beamer.com): Uses Xterm.js to manage digital signage devices from the web dashboard.
+- [**info-beamer hosted**](https://info-beamer.com): Uses xterm.js to manage digital signage devices from the web dashboard.
+- [**Jumpserver**](https://github.com/jumpserver/luna): Jumpserver Luna project, Jumpserver is a bastion server project, Luna use xterm.js for web terminal emulation.
 
 [And much more...](https://github.com/xtermjs/xterm.js/network/dependents)
 
@@ -163,7 +163,7 @@ Do you use xterm.js in your application as well? Please [open a Pull Request](ht
 
 Xterm.js follows a monthly release cycle roughly.
 
-All current and past releases are available on this repo's [Releases page](https://github.com/sourcelair/xterm.js/releases), while a rough roadmap is available by looking through [Milestones](https://github.com/sourcelair/xterm.js/milestones).
+All current and past releases are available on this repo's [Releases page](https://github.com/sourcelair/xterm.js/releases), you can view the [high-level roadmap on the wiki](https://github.com/xtermjs/xterm.js/wiki/Roadmap) and see what we're working on now by looking through [Milestones](https://github.com/sourcelair/xterm.js/milestones).
 
 ## Contributing
 

demo/client.ts

@@ -205,8 +205,7 @@ function initOptions(term: TerminalType): void {
     fontFamily: null,
     fontWeight: ['normal', 'bold', '100', '200', '300', '400', '500', '600', '700', '800', '900'],
     fontWeightBold: ['normal', 'bold', '100', '200', '300', '400', '500', '600', '700', '800', '900'],
-    rendererType: ['dom', 'canvas'],
-    experimentalBufferLineImpl: ['JsArray', 'TypedArray']
+    rendererType: ['dom', 'canvas']
   };
   const options = Object.keys((<any>term)._core.options);
   const booleanOptions = [];

demo/start.js

@@ -5,7 +5,6 @@
  * This file is the entry point for browserify.
  */
 
-const cp = require('child_process');
 const path = require('path');
 const webpack = require('webpack');
 const startServer = require('./server.js');

docker-compose.yml

@@ -7,7 +7,7 @@ services:
     volumes:
       - ./:/usr/src/app
     ports:
-      - ${XTERMJS_PORT:3000}:3000
+      - ${XTERMJS_PORT:-3000}:3000
     command: ["npm", "start"]
 
   watch:

package.json

@@ -1,7 +1,7 @@
 {
   "name": "xterm",
   "description": "Full xterm terminal, in your browser",
-  "version": "3.10.0",
+  "version": "3.12.0",
   "main": "lib/public/Terminal.js",
   "types": "typings/xterm.d.ts",
   "repository": "https://github.com/xtermjs/xterm.js",

src/Buffer.test.ts

@@ -19,8 +19,8 @@ describe('Buffer', () => {
 
   beforeEach(() => {
     terminal = new MockTerminal();
-    terminal.cols = INIT_COLS;
-    terminal.rows = INIT_ROWS;
+    (terminal as any).cols = INIT_COLS;
+    (terminal as any).rows = INIT_ROWS;
     terminal.options.scrollback = 1000;
     buffer = new Buffer(terminal, true);
   });
@@ -108,12 +108,12 @@ describe('Buffer', () => {
 
   describe('resize', () => {
     describe('column size is reduced', () => {
-      it('should not trim the data in the buffer', () => {
+      it('should trim the data in the buffer', () => {
         buffer.fillViewportRows();
         buffer.resize(INIT_COLS / 2, INIT_ROWS);
         assert.equal(buffer.lines.length, INIT_ROWS);
         for (let i = 0; i < INIT_ROWS; i++) {
-          assert.equal(buffer.lines.get(i).length, INIT_COLS);
+          assert.equal(buffer.lines.get(i).length, INIT_COLS / 2);
         }
       });
     });
@@ -233,6 +233,773 @@ describe('Buffer', () => {
         }
       });
     });
+
+    describe('reflow', () => {
+      it('should not wrap empty lines', () => {
+        buffer.fillViewportRows();
+        assert.equal(buffer.lines.length, INIT_ROWS);
+        buffer.resize(INIT_COLS - 5, INIT_ROWS);
+        assert.equal(buffer.lines.length, INIT_ROWS);
+      });
+      it('should shrink row length', () => {
+        buffer.fillViewportRows();
+        buffer.resize(5, 10);
+        assert.equal(buffer.lines.length, 10);
+        assert.equal(buffer.lines.get(0).length, 5);
+        assert.equal(buffer.lines.get(1).length, 5);
+        assert.equal(buffer.lines.get(2).length, 5);
+        assert.equal(buffer.lines.get(3).length, 5);
+        assert.equal(buffer.lines.get(4).length, 5);
+        assert.equal(buffer.lines.get(5).length, 5);
+        assert.equal(buffer.lines.get(6).length, 5);
+        assert.equal(buffer.lines.get(7).length, 5);
+        assert.equal(buffer.lines.get(8).length, 5);
+        assert.equal(buffer.lines.get(9).length, 5);
+      });
+      it('should wrap and unwrap lines', () => {
+        buffer.fillViewportRows();
+        buffer.resize(5, 10);
+        const firstLine = buffer.lines.get(0);
+        for (let i = 0; i < 5; i++) {
+          const code = 'a'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          firstLine.set(i, [null, char, 1, code]);
+        }
+        buffer.y = 1;
+        assert.equal(buffer.lines.get(0).length, 5);
+        assert.equal(buffer.lines.get(0).translateToString(), 'abcde');
+        buffer.resize(1, 10);
+        assert.equal(buffer.lines.length, 10);
+        assert.equal(buffer.lines.get(0).translateToString(), 'a');
+        assert.equal(buffer.lines.get(1).translateToString(), 'b');
+        assert.equal(buffer.lines.get(2).translateToString(), 'c');
+        assert.equal(buffer.lines.get(3).translateToString(), 'd');
+        assert.equal(buffer.lines.get(4).translateToString(), 'e');
+        assert.equal(buffer.lines.get(5).translateToString(), ' ');
+        assert.equal(buffer.lines.get(6).translateToString(), ' ');
+        assert.equal(buffer.lines.get(7).translateToString(), ' ');
+        assert.equal(buffer.lines.get(8).translateToString(), ' ');
+        assert.equal(buffer.lines.get(9).translateToString(), ' ');
+        buffer.resize(5, 10);
+        assert.equal(buffer.lines.length, 10);
+        assert.equal(buffer.lines.get(0).translateToString(), 'abcde');
+        assert.equal(buffer.lines.get(1).translateToString(), '     ');
+        assert.equal(buffer.lines.get(2).translateToString(), '     ');
+        assert.equal(buffer.lines.get(3).translateToString(), '     ');
+        assert.equal(buffer.lines.get(4).translateToString(), '     ');
+        assert.equal(buffer.lines.get(5).translateToString(), '     ');
+        assert.equal(buffer.lines.get(6).translateToString(), '     ');
+        assert.equal(buffer.lines.get(7).translateToString(), '     ');
+        assert.equal(buffer.lines.get(8).translateToString(), '     ');
+        assert.equal(buffer.lines.get(9).translateToString(), '     ');
+      });
+      it('should discard parts of wrapped lines that go out of the scrollback', () => {
+        buffer.fillViewportRows();
+        terminal.options.scrollback = 1;
+        buffer.resize(10, 5);
+        const lastLine = buffer.lines.get(3);
+        for (let i = 0; i < 10; i++) {
+          const code = 'a'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          lastLine.set(i, [null, char, 1, code]);
+        }
+        assert.equal(buffer.lines.length, 5);
+        buffer.y = 4;
+        buffer.resize(2, 5);
+        assert.equal(buffer.y, 4);
+        assert.equal(buffer.ybase, 1);
+        assert.equal(buffer.lines.length, 6);
+        assert.equal(buffer.lines.get(0).translateToString(), 'ab');
+        assert.equal(buffer.lines.get(1).translateToString(), 'cd');
+        assert.equal(buffer.lines.get(2).translateToString(), 'ef');
+        assert.equal(buffer.lines.get(3).translateToString(), 'gh');
+        assert.equal(buffer.lines.get(4).translateToString(), 'ij');
+        assert.equal(buffer.lines.get(5).translateToString(), '  ');
+        buffer.resize(1, 5);
+        assert.equal(buffer.y, 4);
+        assert.equal(buffer.ybase, 1);
+        assert.equal(buffer.lines.length, 6);
+        assert.equal(buffer.lines.get(0).translateToString(), 'f');
+        assert.equal(buffer.lines.get(1).translateToString(), 'g');
+        assert.equal(buffer.lines.get(2).translateToString(), 'h');
+        assert.equal(buffer.lines.get(3).translateToString(), 'i');
+        assert.equal(buffer.lines.get(4).translateToString(), 'j');
+        assert.equal(buffer.lines.get(5).translateToString(), ' ');
+        buffer.resize(10, 5);
+        assert.equal(buffer.y, 1);
+        assert.equal(buffer.ybase, 0);
+        assert.equal(buffer.lines.length, 5);
+        assert.equal(buffer.lines.get(0).translateToString(), 'fghij     ');
+        assert.equal(buffer.lines.get(1).translateToString(), '          ');
+        assert.equal(buffer.lines.get(2).translateToString(), '          ');
+        assert.equal(buffer.lines.get(3).translateToString(), '          ');
+        assert.equal(buffer.lines.get(4).translateToString(), '          ');
+      });
+      it('should remove the correct amount of rows when reflowing larger', () => {
+        // This is a regression test to ensure that successive wrapped lines that are getting
+        // 3+ lines removed on a reflow actually remove the right lines
+        buffer.fillViewportRows();
+        buffer.resize(10, 10);
+        buffer.y = 2;
+        const firstLine = buffer.lines.get(0);
+        const secondLine = buffer.lines.get(1);
+        for (let i = 0; i < 10; i++) {
+          const code = 'a'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          firstLine.set(i, [null, char, 1, code]);
+        }
+        for (let i = 0; i < 10; i++) {
+          const code = '0'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          secondLine.set(i, [null, char, 1, code]);
+        }
+        assert.equal(buffer.lines.length, 10);
+        assert.equal(buffer.lines.get(0).translateToString(), 'abcdefghij');
+        assert.equal(buffer.lines.get(1).translateToString(), '0123456789');
+        for (let i = 2; i < 10; i++) {
+          assert.equal(buffer.lines.get(i).translateToString(), '          ');
+        }
+        buffer.resize(2, 10);
+        assert.equal(buffer.ybase, 1);
+        assert.equal(buffer.lines.length, 11);
+        assert.equal(buffer.lines.get(0).translateToString(), 'ab');
+        assert.equal(buffer.lines.get(1).translateToString(), 'cd');
+        assert.equal(buffer.lines.get(2).translateToString(), 'ef');
+        assert.equal(buffer.lines.get(3).translateToString(), 'gh');
+        assert.equal(buffer.lines.get(4).translateToString(), 'ij');
+        assert.equal(buffer.lines.get(5).translateToString(), '01');
+        assert.equal(buffer.lines.get(6).translateToString(), '23');
+        assert.equal(buffer.lines.get(7).translateToString(), '45');
+        assert.equal(buffer.lines.get(8).translateToString(), '67');
+        assert.equal(buffer.lines.get(9).translateToString(), '89');
+        assert.equal(buffer.lines.get(10).translateToString(), '  ');
+        buffer.resize(10, 10);
+        assert.equal(buffer.ybase, 0);
+        assert.equal(buffer.lines.length, 10);
+        assert.equal(buffer.lines.get(0).translateToString(), 'abcdefghij');
+        assert.equal(buffer.lines.get(1).translateToString(), '0123456789');
+        for (let i = 2; i < 10; i++) {
+          assert.equal(buffer.lines.get(i).translateToString(), '          ');
+        }
+      });
+      it('should transfer combined char data over to reflowed lines', () => {
+        buffer.fillViewportRows();
+        buffer.resize(4, 3);
+        buffer.y = 2;
+        const firstLine = buffer.lines.get(0);
+        firstLine.set(0, [ null, 'a', 1, 'a'.charCodeAt(0) ]);
+        firstLine.set(1, [ null, 'b', 1, 'b'.charCodeAt(0) ]);
+        firstLine.set(2, [ null, 'c', 1, 'c'.charCodeAt(0) ]);
+        firstLine.set(3, [ null, '😁', 1, '😁'.charCodeAt(0) ]);
+        assert.equal(buffer.lines.length, 3);
+        assert.equal(buffer.lines.get(0).translateToString(), 'abc😁');
+        assert.equal(buffer.lines.get(1).translateToString(), '    ');
+        buffer.resize(2, 3);
+        assert.equal(buffer.lines.get(0).translateToString(), 'ab');
+        assert.equal(buffer.lines.get(1).translateToString(), 'c😁');
+      });
+      it('should adjust markers when reflowing', () => {
+        buffer.fillViewportRows();
+        buffer.resize(10, 16);
+        for (let i = 0; i < 10; i++) {
+          const code = 'a'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          buffer.lines.get(0).set(i, [null, char, 1, code]);
+        }
+        for (let i = 0; i < 10; i++) {
+          const code = '0'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          buffer.lines.get(1).set(i, [null, char, 1, code]);
+        }
+        for (let i = 0; i < 10; i++) {
+          const code = 'k'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          buffer.lines.get(2).set(i, [null, char, 1, code]);
+        }
+        buffer.y = 3;
+        // Buffer:
+        // abcdefghij
+        // 0123456789
+        // abcdefghij
+        const firstMarker = buffer.addMarker(0);
+        const secondMarker = buffer.addMarker(1);
+        const thirdMarker = buffer.addMarker(2);
+        assert.equal(buffer.lines.get(0).translateToString(), 'abcdefghij');
+        assert.equal(buffer.lines.get(1).translateToString(), '0123456789');
+        assert.equal(buffer.lines.get(2).translateToString(), 'klmnopqrst');
+        assert.equal(firstMarker.line, 0);
+        assert.equal(secondMarker.line, 1);
+        assert.equal(thirdMarker.line, 2);
+        buffer.resize(2, 16);
+        assert.equal(buffer.lines.get(0).translateToString(), 'ab');
+        assert.equal(buffer.lines.get(1).translateToString(), 'cd');
+        assert.equal(buffer.lines.get(2).translateToString(), 'ef');
+        assert.equal(buffer.lines.get(3).translateToString(), 'gh');
+        assert.equal(buffer.lines.get(4).translateToString(), 'ij');
+        assert.equal(buffer.lines.get(5).translateToString(), '01');
+        assert.equal(buffer.lines.get(6).translateToString(), '23');
+        assert.equal(buffer.lines.get(7).translateToString(), '45');
+        assert.equal(buffer.lines.get(8).translateToString(), '67');
+        assert.equal(buffer.lines.get(9).translateToString(), '89');
+        assert.equal(buffer.lines.get(10).translateToString(), 'kl');
+        assert.equal(buffer.lines.get(11).translateToString(), 'mn');
+        assert.equal(buffer.lines.get(12).translateToString(), 'op');
+        assert.equal(buffer.lines.get(13).translateToString(), 'qr');
+        assert.equal(buffer.lines.get(14).translateToString(), 'st');
+        assert.equal(firstMarker.line, 0, 'first marker should remain unchanged');
+        assert.equal(secondMarker.line, 5, 'second marker should be shifted since the first line wrapped');
+        assert.equal(thirdMarker.line, 10, 'third marker should be shifted since the first and second lines wrapped');
+        buffer.resize(10, 16);
+        assert.equal(buffer.lines.get(0).translateToString(), 'abcdefghij');
+        assert.equal(buffer.lines.get(1).translateToString(), '0123456789');
+        assert.equal(buffer.lines.get(2).translateToString(), 'klmnopqrst');
+        assert.equal(firstMarker.line, 0, 'first marker should remain unchanged');
+        assert.equal(secondMarker.line, 1, 'second marker should be restored to it\'s original line');
+        assert.equal(thirdMarker.line, 2, 'third marker should be restored to it\'s original line');
+        assert.equal(firstMarker.isDisposed, false);
+        assert.equal(secondMarker.isDisposed, false);
+        assert.equal(thirdMarker.isDisposed, false);
+      });
+      it('should dispose markers whose rows are trimmed during a reflow', () => {
+        buffer.fillViewportRows();
+        terminal.options.scrollback = 1;
+        buffer.resize(10, 11);
+        for (let i = 0; i < 10; i++) {
+          const code = 'a'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          buffer.lines.get(0).set(i, [null, char, 1, code]);
+        }
+        for (let i = 0; i < 10; i++) {
+          const code = '0'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          buffer.lines.get(1).set(i, [null, char, 1, code]);
+        }
+        for (let i = 0; i < 10; i++) {
+          const code = 'k'.charCodeAt(0) + i;
+          const char = String.fromCharCode(code);
+          buffer.lines.get(2).set(i, [null, char, 1, code]);
+        }
+        buffer.y = 10;
+        // Buffer:
+        // abcdefghij
+        // 0123456789
+        // abcdefghij
+        const firstMarker = buffer.addMarker(0);
+        const secondMarker = buffer.addMarker(1);
+        const thirdMarker = buffer.addMarker(2);
+        buffer.y = 3;
+        assert.equal(buffer.lines.get(0).translateToString(), 'abcdefghij');
+        assert.equal(buffer.lines.get(1).translateToString(), '0123456789');
+        assert.equal(buffer.lines.get(2).translateToString(), 'klmnopqrst');
+        assert.equal(firstMarker.line, 0);
+        assert.equal(secondMarker.line, 1);
+        assert.equal(thirdMarker.line, 2);
+        buffer.resize(2, 11);
+        assert.equal(buffer.lines.get(0).translateToString(), 'ij');
+        assert.equal(buffer.lines.get(1).translateToString(), '01');
+        assert.equal(buffer.lines.get(2).translateToString(), '23');
+        assert.equal(buffer.lines.get(3).translateToString(), '45');
+        assert.equal(buffer.lines.get(4).translateToString(), '67');
+        assert.equal(buffer.lines.get(5).translateToString(), '89');
+        assert.equal(buffer.lines.get(6).translateToString(), 'kl');
+        assert.equal(buffer.lines.get(7).translateToString(), 'mn');
+        assert.equal(buffer.lines.get(8).translateToString(), 'op');
+        assert.equal(buffer.lines.get(9).translateToString(), 'qr');
+        assert.equal(buffer.lines.get(10).translateToString(), 'st');
+        assert.equal(secondMarker.line, 1, 'second marker should remain the same as it was shifted 4 and trimmed 4');
+        assert.equal(thirdMarker.line, 6, 'third marker should be shifted since the first and second lines wrapped');
+        assert.equal(firstMarker.isDisposed, true, 'first marker was trimmed');
+        assert.equal(secondMarker.isDisposed, false);
+        assert.equal(thirdMarker.isDisposed, false);
+        buffer.resize(10, 11);
+        assert.equal(buffer.lines.get(0).translateToString(), 'ij        ');
+        assert.equal(buffer.lines.get(1).translateToString(), '0123456789');
+        assert.equal(buffer.lines.get(2).translateToString(), 'klmnopqrst');
+        assert.equal(secondMarker.line, 1, 'second marker should be restored');
+        assert.equal(thirdMarker.line, 2, 'third marker should be restored');
+      });
+      it('should wrap wide characters correctly when reflowing larger', () => {
+        buffer.fillViewportRows();
+        buffer.resize(12, 10);
+        buffer.y = 2;
+        for (let i = 0; i < 12; i += 4) {
+          buffer.lines.get(0).set(i, [null, '汉', 2, '汉'.charCodeAt(0)]);
+          buffer.lines.get(1).set(i, [null, '汉', 2, '汉'.charCodeAt(0)]);
+        }
+        for (let i = 2; i < 12; i += 4) {
+          buffer.lines.get(0).set(i, [null, '语', 2, '语'.charCodeAt(0)]);
+          buffer.lines.get(1).set(i, [null, '语', 2, '语'.charCodeAt(0)]);
+        }
+        for (let i = 1; i < 12; i += 2) {
+          buffer.lines.get(0).set(i, [null, '', 0, undefined]);
+          buffer.lines.get(1).set(i, [null, '', 0, undefined]);
+        }
+        buffer.lines.get(1).isWrapped = true;
+        // Buffer:
+        // 汉语汉语汉语 (wrapped)
+        // 汉语汉语汉语
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语汉语');
+        assert.equal(buffer.lines.get(1).translateToString(true), '汉语汉语汉语');
+        buffer.resize(13, 10);
+        assert.equal(buffer.ybase, 0);
+        assert.equal(buffer.lines.length, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语汉语');
+        assert.equal(buffer.lines.get(0).translateToString(false), '汉语汉语汉语 ');
+        assert.equal(buffer.lines.get(1).translateToString(true), '汉语汉语汉语');
+        assert.equal(buffer.lines.get(1).translateToString(false), '汉语汉语汉语 ');
+        buffer.resize(14, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语汉语汉');
+        assert.equal(buffer.lines.get(0).translateToString(false), '汉语汉语汉语汉');
+        assert.equal(buffer.lines.get(1).translateToString(true), '语汉语汉语');
+        assert.equal(buffer.lines.get(1).translateToString(false), '语汉语汉语    ');
+      });
+      it('should wrap wide characters correctly when reflowing smaller', () => {
+        buffer.fillViewportRows();
+        buffer.resize(12, 10);
+        buffer.y = 2;
+        for (let i = 0; i < 12; i += 4) {
+          buffer.lines.get(0).set(i, [null, '汉', 2, '汉'.charCodeAt(0)]);
+          buffer.lines.get(1).set(i, [null, '汉', 2, '汉'.charCodeAt(0)]);
+        }
+        for (let i = 2; i < 12; i += 4) {
+          buffer.lines.get(0).set(i, [null, '语', 2, '语'.charCodeAt(0)]);
+          buffer.lines.get(1).set(i, [null, '语', 2, '语'.charCodeAt(0)]);
+        }
+        for (let i = 1; i < 12; i += 2) {
+          buffer.lines.get(0).set(i, [null, '', 0, undefined]);
+          buffer.lines.get(1).set(i, [null, '', 0, undefined]);
+        }
+        buffer.lines.get(1).isWrapped = true;
+        // Buffer:
+        // 汉语汉语汉语 (wrapped)
+        // 汉语汉语汉语
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语汉语');
+        assert.equal(buffer.lines.get(1).translateToString(true), '汉语汉语汉语');
+        buffer.resize(11, 10);
+        assert.equal(buffer.ybase, 0);
+        assert.equal(buffer.lines.length, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语汉');
+        assert.equal(buffer.lines.get(1).translateToString(true), '语汉语汉语');
+        assert.equal(buffer.lines.get(2).translateToString(true), '汉语');
+        buffer.resize(10, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语汉');
+        assert.equal(buffer.lines.get(1).translateToString(true), '语汉语汉语');
+        assert.equal(buffer.lines.get(2).translateToString(true), '汉语');
+        buffer.resize(9, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语');
+        assert.equal(buffer.lines.get(1).translateToString(true), '汉语汉语');
+        assert.equal(buffer.lines.get(2).translateToString(true), '汉语汉语');
+        buffer.resize(8, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉语');
+        assert.equal(buffer.lines.get(1).translateToString(true), '汉语汉语');
+        assert.equal(buffer.lines.get(2).translateToString(true), '汉语汉语');
+        buffer.resize(7, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉');
+        assert.equal(buffer.lines.get(1).translateToString(true), '语汉语');
+        assert.equal(buffer.lines.get(2).translateToString(true), '汉语汉');
+        assert.equal(buffer.lines.get(3).translateToString(true), '语汉语');
+        buffer.resize(6, 10);
+        assert.equal(buffer.lines.get(0).translateToString(true), '汉语汉');
+        assert.equal(buffer.lines.get(1).translateToString(true), '语汉语');
+        assert.equal(buffer.lines.get(2).translateToString(true), '汉语汉');
+        assert.equal(buffer.lines.get(3).translateToString(true), '语汉语');
+      });
+
+      describe('reflowLarger cases', () => {
+        beforeEach(() => {
+          // Setup buffer state:
+          // 'ab'
+          // 'cd' (wrapped)
+          // 'ef'
+          // 'gh' (wrapped)
+          // 'ij'
+          // 'kl' (wrapped)
+          // '  '
+          // '  '
+          // '  '
+          // '  '
+          buffer.fillViewportRows();
+          buffer.resize(2, 10);
+          buffer.lines.get(0).set(0, [null, 'a', 1, 'a'.charCodeAt(0)]);
+          buffer.lines.get(0).set(1, [null, 'b', 1, 'b'.charCodeAt(0)]);
+          buffer.lines.get(1).set(0, [null, 'c', 1, 'c'.charCodeAt(0)]);
+          buffer.lines.get(1).set(1, [null, 'd', 1, 'd'.charCodeAt(0)]);
+          buffer.lines.get(1).isWrapped = true;
+          buffer.lines.get(2).set(0, [null, 'e', 1, 'e'.charCodeAt(0)]);
+          buffer.lines.get(2).set(1, [null, 'f', 1, 'f'.charCodeAt(0)]);
+          buffer.lines.get(3).set(0, [null, 'g', 1, 'g'.charCodeAt(0)]);
+          buffer.lines.get(3).set(1, [null, 'h', 1, 'h'.charCodeAt(0)]);
+          buffer.lines.get(3).isWrapped = true;
+          buffer.lines.get(4).set(0, [null, 'i', 1, 'i'.charCodeAt(0)]);
+          buffer.lines.get(4).set(1, [null, 'j', 1, 'j'.charCodeAt(0)]);
+          buffer.lines.get(5).set(0, [null, 'k', 1, 'k'.charCodeAt(0)]);
+          buffer.lines.get(5).set(1, [null, 'l', 1, 'l'.charCodeAt(0)]);
+          buffer.lines.get(5).isWrapped = true;
+        });
+        describe('viewport not yet filled', () => {
+          it('should move the cursor up and add empty lines', () => {
+            buffer.y = 6;
+            buffer.resize(4, 10);
+            assert.equal(buffer.y, 3);
+            assert.equal(buffer.ydisp, 0);
+            assert.equal(buffer.ybase, 0);
+            assert.equal(buffer.lines.length, 10);
+            assert.equal(buffer.lines.get(0).translateToString(), 'abcd');
+            assert.equal(buffer.lines.get(1).translateToString(), 'efgh');
+            assert.equal(buffer.lines.get(2).translateToString(), 'ijkl');
+            for (let i = 3; i < 10; i++) {
+              assert.equal(buffer.lines.get(i).translateToString(), '    ');
+            }
+            const wrappedLines: number[] = [];
+            for (let i = 0; i < buffer.lines.length; i++) {
+              assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+            }
+          });
+        });
+        describe('viewport filled, scrollback remaining', () => {
+          beforeEach(() => {
+            buffer.y = 9;
+          });
+          describe('ybase === 0', () => {
+            it('should move the cursor up and add empty lines', () => {
+              buffer.resize(4, 10);
+              assert.equal(buffer.y, 6);
+              assert.equal(buffer.ydisp, 0);
+              assert.equal(buffer.ybase, 0);
+              assert.equal(buffer.lines.length, 10);
+              assert.equal(buffer.lines.get(0).translateToString(), 'abcd');
+              assert.equal(buffer.lines.get(1).translateToString(), 'efgh');
+              assert.equal(buffer.lines.get(2).translateToString(), 'ijkl');
+              for (let i = 3; i < 10; i++) {
+                assert.equal(buffer.lines.get(i).translateToString(), '    ');
+              }
+              const wrappedLines: number[] = [];
+              for (let i = 0; i < buffer.lines.length; i++) {
+                assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+              }
+            });
+          });
+          describe('ybase !== 0', () => {
+            beforeEach(() => {
+              // Add 10 empty rows to start
+              for (let i = 0; i < 10; i++) {
+                buffer.lines.splice(0, 0, buffer.getBlankLine(DEFAULT_ATTR));
+              }
+              buffer.ybase = 10;
+            });
+            describe('&& ydisp === ybase', () => {
+              it('should adjust the viewport and keep ydisp = ybase', () => {
+                buffer.ydisp = 10;
+                buffer.resize(4, 10);
+                assert.equal(buffer.y, 9);
+                assert.equal(buffer.ydisp, 7);
+                assert.equal(buffer.ybase, 7);
+                assert.equal(buffer.lines.length, 17);
+                for (let i = 0; i < 10; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                assert.equal(buffer.lines.get(10).translateToString(), 'abcd');
+                assert.equal(buffer.lines.get(11).translateToString(), 'efgh');
+                assert.equal(buffer.lines.get(12).translateToString(), 'ijkl');
+                for (let i = 13; i < 17; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                const wrappedLines: number[] = [];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+            describe('&& ydisp !== ybase', () => {
+              it('should keep ydisp at the same value', () => {
+                buffer.ydisp = 5;
+                buffer.resize(4, 10);
+                assert.equal(buffer.y, 9);
+                assert.equal(buffer.ydisp, 5);
+                assert.equal(buffer.ybase, 7);
+                assert.equal(buffer.lines.length, 17);
+                for (let i = 0; i < 10; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                assert.equal(buffer.lines.get(10).translateToString(), 'abcd');
+                assert.equal(buffer.lines.get(11).translateToString(), 'efgh');
+                assert.equal(buffer.lines.get(12).translateToString(), 'ijkl');
+                for (let i = 13; i < 17; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                const wrappedLines: number[] = [];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+          });
+        });
+        describe('viewport filled, no scrollback remaining', () => {
+          // ybase === 0 doesn't make sense here as scrollback=0 isn't really supported
+          describe('ybase !== 0', () => {
+            beforeEach(() => {
+              terminal.options.scrollback = 10;
+              // Add 10 empty rows to start
+              for (let i = 0; i < 10; i++) {
+                buffer.lines.splice(0, 0, buffer.getBlankLine(DEFAULT_ATTR));
+              }
+              buffer.y = 9;
+              buffer.ybase = 10;
+            });
+            describe('&& ydisp === ybase', () => {
+              it('should trim lines and keep ydisp = ybase', () => {
+                buffer.ydisp = 10;
+                buffer.resize(4, 10);
+                assert.equal(buffer.y, 9);
+                assert.equal(buffer.ydisp, 7);
+                assert.equal(buffer.ybase, 7);
+                assert.equal(buffer.lines.length, 17);
+                for (let i = 0; i < 10; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                assert.equal(buffer.lines.get(10).translateToString(), 'abcd');
+                assert.equal(buffer.lines.get(11).translateToString(), 'efgh');
+                assert.equal(buffer.lines.get(12).translateToString(), 'ijkl');
+                for (let i = 13; i < 17; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                const wrappedLines: number[] = [];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+            describe('&& ydisp !== ybase', () => {
+              it('should trim lines and not change ydisp', () => {
+                buffer.ydisp = 5;
+                buffer.resize(4, 10);
+                assert.equal(buffer.y, 9);
+                assert.equal(buffer.ydisp, 5);
+                assert.equal(buffer.ybase, 7);
+                assert.equal(buffer.lines.length, 17);
+                for (let i = 0; i < 10; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                assert.equal(buffer.lines.get(10).translateToString(), 'abcd');
+                assert.equal(buffer.lines.get(11).translateToString(), 'efgh');
+                assert.equal(buffer.lines.get(12).translateToString(), 'ijkl');
+                for (let i = 13; i < 17; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '    ');
+                }
+                const wrappedLines: number[] = [];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+          });
+        });
+      });
+      describe('reflowSmaller cases', () => {
+        beforeEach(() => {
+          // Setup buffer state:
+          // 'abcd'
+          // 'efgh' (wrapped)
+          // 'ijkl'
+          // '    '
+          // '    '
+          // '    '
+          // '    '
+          // '    '
+          // '    '
+          // '    '
+          buffer.fillViewportRows();
+          buffer.resize(4, 10);
+          buffer.lines.get(0).set(0, [null, 'a', 1, 'a'.charCodeAt(0)]);
+          buffer.lines.get(0).set(1, [null, 'b', 1, 'b'.charCodeAt(0)]);
+          buffer.lines.get(0).set(2, [null, 'c', 1, 'c'.charCodeAt(0)]);
+          buffer.lines.get(0).set(3, [null, 'd', 1, 'd'.charCodeAt(0)]);
+          buffer.lines.get(1).set(0, [null, 'e', 1, 'e'.charCodeAt(0)]);
+          buffer.lines.get(1).set(1, [null, 'f', 1, 'f'.charCodeAt(0)]);
+          buffer.lines.get(1).set(2, [null, 'g', 1, 'g'.charCodeAt(0)]);
+          buffer.lines.get(1).set(3, [null, 'h', 1, 'h'.charCodeAt(0)]);
+          buffer.lines.get(2).set(0, [null, 'i', 1, 'i'.charCodeAt(0)]);
+          buffer.lines.get(2).set(1, [null, 'j', 1, 'j'.charCodeAt(0)]);
+          buffer.lines.get(2).set(2, [null, 'k', 1, 'k'.charCodeAt(0)]);
+          buffer.lines.get(2).set(3, [null, 'l', 1, 'l'.charCodeAt(0)]);
+        });
+        describe('viewport not yet filled', () => {
+          it('should move the cursor down', () => {
+            buffer.y = 3;
+            buffer.resize(2, 10);
+            assert.equal(buffer.y, 6);
+            assert.equal(buffer.ydisp, 0);
+            assert.equal(buffer.ybase, 0);
+            assert.equal(buffer.lines.length, 10);
+            assert.equal(buffer.lines.get(0).translateToString(), 'ab');
+            assert.equal(buffer.lines.get(1).translateToString(), 'cd');
+            assert.equal(buffer.lines.get(2).translateToString(), 'ef');
+            assert.equal(buffer.lines.get(3).translateToString(), 'gh');
+            assert.equal(buffer.lines.get(4).translateToString(), 'ij');
+            assert.equal(buffer.lines.get(5).translateToString(), 'kl');
+            for (let i = 6; i < 10; i++) {
+              assert.equal(buffer.lines.get(i).translateToString(), '  ');
+            }
+            const wrappedLines = [1, 3, 5];
+            for (let i = 0; i < buffer.lines.length; i++) {
+              assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+            }
+          });
+        });
+        describe('viewport filled, scrollback remaining', () => {
+          beforeEach(() => {
+            buffer.y = 9;
+          });
+          describe('ybase === 0', () => {
+            it('should trim the top', () => {
+              buffer.resize(2, 10);
+              assert.equal(buffer.y, 9);
+              assert.equal(buffer.ydisp, 3);
+              assert.equal(buffer.ybase, 3);
+              assert.equal(buffer.lines.length, 13);
+              assert.equal(buffer.lines.get(0).translateToString(), 'ab');
+              assert.equal(buffer.lines.get(1).translateToString(), 'cd');
+              assert.equal(buffer.lines.get(2).translateToString(), 'ef');
+              assert.equal(buffer.lines.get(3).translateToString(), 'gh');
+              assert.equal(buffer.lines.get(4).translateToString(), 'ij');
+              assert.equal(buffer.lines.get(5).translateToString(), 'kl');
+              for (let i = 6; i < 13; i++) {
+                assert.equal(buffer.lines.get(i).translateToString(), '  ');
+              }
+              const wrappedLines = [1, 3, 5];
+              for (let i = 0; i < buffer.lines.length; i++) {
+                assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+              }
+            });
+          });
+          describe('ybase !== 0', () => {
+            beforeEach(() => {
+              // Add 10 empty rows to start
+              for (let i = 0; i < 10; i++) {
+                buffer.lines.splice(0, 0, buffer.getBlankLine(DEFAULT_ATTR));
+              }
+              buffer.ybase = 10;
+            });
+            describe('&& ydisp === ybase', () => {
+              it('should adjust the viewport and keep ydisp = ybase', () => {
+                buffer.ydisp = 10;
+                buffer.resize(2, 10);
+                assert.equal(buffer.ydisp, 13);
+                assert.equal(buffer.ybase, 13);
+                assert.equal(buffer.lines.length, 23);
+                for (let i = 0; i < 10; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                assert.equal(buffer.lines.get(10).translateToString(), 'ab');
+                assert.equal(buffer.lines.get(11).translateToString(), 'cd');
+                assert.equal(buffer.lines.get(12).translateToString(), 'ef');
+                assert.equal(buffer.lines.get(13).translateToString(), 'gh');
+                assert.equal(buffer.lines.get(14).translateToString(), 'ij');
+                assert.equal(buffer.lines.get(15).translateToString(), 'kl');
+                for (let i = 16; i < 23; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                const wrappedLines = [11, 13, 15];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+            describe('&& ydisp !== ybase', () => {
+              it('should keep ydisp at the same value', () => {
+                buffer.ydisp = 5;
+                buffer.resize(2, 10);
+                assert.equal(buffer.ydisp, 5);
+                assert.equal(buffer.ybase, 13);
+                assert.equal(buffer.lines.length, 23);
+                for (let i = 0; i < 10; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                assert.equal(buffer.lines.get(10).translateToString(), 'ab');
+                assert.equal(buffer.lines.get(11).translateToString(), 'cd');
+                assert.equal(buffer.lines.get(12).translateToString(), 'ef');
+                assert.equal(buffer.lines.get(13).translateToString(), 'gh');
+                assert.equal(buffer.lines.get(14).translateToString(), 'ij');
+                assert.equal(buffer.lines.get(15).translateToString(), 'kl');
+                for (let i = 16; i < 23; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                const wrappedLines = [11, 13, 15];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+          });
+        });
+        describe('viewport filled, no scrollback remaining', () => {
+          // ybase === 0 doesn't make sense here as scrollback=0 isn't really supported
+          describe('ybase !== 0', () => {
+            beforeEach(() => {
+              terminal.options.scrollback = 10;
+              // Add 10 empty rows to start
+              for (let i = 0; i < 10; i++) {
+                buffer.lines.splice(0, 0, buffer.getBlankLine(DEFAULT_ATTR));
+              }
+              buffer.ybase = 10;
+            });
+            describe('&& ydisp === ybase', () => {
+              it('should trim lines and keep ydisp = ybase', () => {
+                buffer.ydisp = 10;
+                buffer.y = 13;
+                buffer.resize(2, 10);
+                assert.equal(buffer.ydisp, 10);
+                assert.equal(buffer.ybase, 10);
+                assert.equal(buffer.lines.length, 20);
+                for (let i = 0; i < 7; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                assert.equal(buffer.lines.get(7).translateToString(), 'ab');
+                assert.equal(buffer.lines.get(8).translateToString(), 'cd');
+                assert.equal(buffer.lines.get(9).translateToString(), 'ef');
+                assert.equal(buffer.lines.get(10).translateToString(), 'gh');
+                assert.equal(buffer.lines.get(11).translateToString(), 'ij');
+                assert.equal(buffer.lines.get(12).translateToString(), 'kl');
+                for (let i = 13; i < 20; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                const wrappedLines = [8, 10, 12];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+            describe('&& ydisp !== ybase', () => {
+              it('should trim lines and not change ydisp', () => {
+                buffer.ydisp = 5;
+                buffer.y = 13;
+                buffer.resize(2, 10);
+                assert.equal(buffer.ydisp, 5);
+                assert.equal(buffer.ybase, 10);
+                assert.equal(buffer.lines.length, 20);
+                for (let i = 0; i < 7; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                assert.equal(buffer.lines.get(7).translateToString(), 'ab');
+                assert.equal(buffer.lines.get(8).translateToString(), 'cd');
+                assert.equal(buffer.lines.get(9).translateToString(), 'ef');
+                assert.equal(buffer.lines.get(10).translateToString(), 'gh');
+                assert.equal(buffer.lines.get(11).translateToString(), 'ij');
+                assert.equal(buffer.lines.get(12).translateToString(), 'kl');
+                for (let i = 13; i < 20; i++) {
+                  assert.equal(buffer.lines.get(i).translateToString(), '  ');
+                }
+                const wrappedLines = [8, 10, 12];
+                for (let i = 0; i < buffer.lines.length; i++) {
+                  assert.equal(buffer.lines.get(i).isWrapped, wrappedLines.indexOf(i) !== -1, `line ${i} isWrapped must equal ${wrappedLines.indexOf(i) !== -1}`);
+                }
+              });
+            });
+          });
+        });
+      });
+    });
   });
 
   describe('buffer marked to have no scrollback', () => {
@@ -511,7 +1278,7 @@ describe('Buffer', () => {
       const s = terminal.buffer.iterator(true).next().content;
       assert.equal(input, s);
       for (let i = 10; i < input.length; ++i) {
-        const bufferIndex = terminal.buffer.stringIndexToBufferIndex(0, i);
+        const bufferIndex = terminal.buffer.stringIndexToBufferIndex(0, i, true);
         const j = (i - 0) << 1;
         assert.deepEqual([(j / terminal.cols) | 0, j % terminal.cols], bufferIndex);
       }
@@ -523,7 +1290,7 @@ describe('Buffer', () => {
       const s = terminal.buffer.iterator(true).next().content;
       assert.equal(input, s);
       for (let i = 0; i < input.length; ++i) {
-        const bufferIndex = terminal.buffer.stringIndexToBufferIndex(0, i);
+        const bufferIndex = terminal.buffer.stringIndexToBufferIndex(0, i, true);
         assert.equal(input[i], terminal.buffer.lines.get(bufferIndex[0]).get(bufferIndex[1])[CHAR_DATA_CHAR_INDEX]);
       }
     });
@@ -535,7 +1302,7 @@ describe('Buffer', () => {
       const s = terminal.buffer.iterator(true).next().content;
       assert.equal(input, s);
       for (let i = 0; i < input.length; ++i) {
-        const bufferIndex = terminal.buffer.stringIndexToBufferIndex(0, i);
+        const bufferIndex = terminal.buffer.stringIndexToBufferIndex(0, i, true);
         assert.equal(
           (!(i % 3))
             ? input[i]
@@ -545,6 +1312,13 @@ describe('Buffer', () => {
           terminal.buffer.lines.get(bufferIndex[0]).get(bufferIndex[1])[CHAR_DATA_CHAR_INDEX]);
       }
     });
+
+    it('should handle \t in lines correctly', () => {
+      const input = '\thttps://google.de';
+      terminal.writeSync(input);
+      const s = terminal.buffer.iterator(true).next().content;
+      assert.equal(s, Array(terminal.getOption('tabStopWidth') + 1).join(' ') + 'https://google.de');
+    });
   });
   describe('BufferStringIterator', function(): void {
     it('iterator does not overflow buffer limits', function(): void {

src/Buffer.ts

@@ -3,12 +3,13 @@
  * @license MIT
  */
 
-import { CircularList } from './common/CircularList';
-import { CharData, ITerminal, IBuffer, IBufferLine, BufferIndex, IBufferStringIterator, IBufferStringIteratorResult, IBufferLineConstructor } from './Types';
-import { EventEmitter } from './common/EventEmitter';
 import { IMarker } from 'xterm';
-import { BufferLine, BufferLineJSArray } from './BufferLine';
+import { BufferLine } from './BufferLine';
+import { reflowLargerApplyNewLayout, reflowLargerCreateNewLayout, reflowLargerGetLinesToRemove, reflowSmallerGetNewLineLengths } from './BufferReflow';
+import { CircularList, IDeleteEvent, IInsertEvent } from './common/CircularList';
+import { EventEmitter } from './common/EventEmitter';
 import { DEFAULT_COLOR } from './renderer/atlas/Types';
+import { BufferIndex, CharData, IBuffer, IBufferLine, IBufferStringIterator, IBufferStringIteratorResult, ITerminal } from './Types';
 
 export const DEFAULT_ATTR = (0 << 18) | (DEFAULT_COLOR << 9) | (256 << 0);
 export const CHAR_DATA_ATTR_INDEX = 0;
@@ -25,6 +26,8 @@ export const WHITESPACE_CELL_CHAR = ' ';
 export const WHITESPACE_CELL_WIDTH = 1;
 export const WHITESPACE_CELL_CODE = 32;
 
+export const FILL_CHAR_DATA: CharData = [DEFAULT_ATTR, NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE];
+
 /**
  * This class represents a terminal buffer (an internal state of the terminal), where the
  * following information is stored (in high-level):
@@ -45,7 +48,8 @@ export class Buffer implements IBuffer {
   public savedX: number;
   public savedCurAttr: number;
   public markers: Marker[] = [];
-  private _bufferLineConstructor: IBufferLineConstructor;
+  private _cols: number;
+  private _rows: number;
 
   /**
    * Create a new Buffer.
@@ -57,48 +61,24 @@ export class Buffer implements IBuffer {
     private _terminal: ITerminal,
     private _hasScrollback: boolean
   ) {
+    this._cols = this._terminal.cols;
+    this._rows = this._terminal.rows;
     this.clear();
   }
 
-  public setBufferLineFactory(type: string): void {
-    if (type === 'JsArray') {
-      if (this._bufferLineConstructor !== BufferLineJSArray) {
-        this._bufferLineConstructor = BufferLineJSArray;
-        this._recreateLines();
-      }
-    } else {
-      if (this._bufferLineConstructor !== BufferLine) {
-        this._bufferLineConstructor = BufferLine;
-        this._recreateLines();
-      }
-    }
-  }
-
-  private _recreateLines(): void {
-    if (!this.lines) return;
-    for (let i = 0; i < this.lines.length; ++i) {
-      const oldLine = this.lines.get(i);
-      const newLine = new this._bufferLineConstructor(oldLine.length);
-      for (let j = 0; j < oldLine.length; ++j) {
-        newLine.set(j, oldLine.get(j));
-      }
-      this.lines.set(i, newLine);
-    }
-  }
-
   public getBlankLine(attr: number, isWrapped?: boolean): IBufferLine {
     const fillCharData: CharData = [attr, NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE];
-    return new this._bufferLineConstructor(this._terminal.cols, fillCharData, isWrapped);
+    return new BufferLine(this._cols, fillCharData, isWrapped);
   }
 
   public get hasScrollback(): boolean {
-    return this._hasScrollback && this.lines.maxLength > this._terminal.rows;
+    return this._hasScrollback && this.lines.maxLength > this._rows;
   }
 
   public get isCursorInViewport(): boolean {
     const absoluteY = this.ybase + this.y;
     const relativeY = absoluteY - this.ydisp;
-    return (relativeY >= 0 && relativeY < this._terminal.rows);
+    return (relativeY >= 0 && relativeY < this._rows);
   }
 
   /**
@@ -124,7 +104,7 @@ export class Buffer implements IBuffer {
       if (fillAttr === undefined) {
         fillAttr = DEFAULT_ATTR;
       }
-      let i = this._terminal.rows;
+      let i = this._rows;
       while (i--) {
         this.lines.push(this.getBlankLine(fillAttr));
       }
@@ -135,14 +115,13 @@ export class Buffer implements IBuffer {
    * Clears the buffer to it's initial state, discarding all previous data.
    */
   public clear(): void {
-    this.setBufferLineFactory(this._terminal.options.experimentalBufferLineImpl);
     this.ydisp = 0;
     this.ybase = 0;
     this.y = 0;
     this.x = 0;
-    this.lines = new CircularList<IBufferLine>(this._getCorrectBufferLength(this._terminal.rows));
+    this.lines = new CircularList<IBufferLine>(this._getCorrectBufferLength(this._rows));
     this.scrollTop = 0;
-    this.scrollBottom = this._terminal.rows - 1;
+    this.scrollBottom = this._rows - 1;
     this.setupTabStops();
   }
 
@@ -162,18 +141,17 @@ export class Buffer implements IBuffer {
     // The following adjustments should only happen if the buffer has been
     // initialized/filled.
     if (this.lines.length > 0) {
-      // Deal with columns increasing (we don't do anything when columns reduce)
-      if (this._terminal.cols < newCols) {
-        const ch: CharData = [DEFAULT_ATTR, NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE]; // does xterm use the default attr?
+      // Deal with columns increasing (reducing needs to happen after reflow)
+      if (this._cols < newCols) {
         for (let i = 0; i < this.lines.length; i++) {
-          this.lines.get(i).resize(newCols, ch);
+          this.lines.get(i).resize(newCols, FILL_CHAR_DATA);
         }
       }
 
       // Resize rows in both directions as needed
       let addToY = 0;
-      if (this._terminal.rows < newRows) {
-        for (let y = this._terminal.rows; y < newRows; y++) {
+      if (this._rows < newRows) {
+        for (let y = this._rows; y < newRows; y++) {
           if (this.lines.length < newRows + this.ybase) {
             if (this.ybase > 0 && this.lines.length <= this.ybase + this.y + addToY + 1) {
               // There is room above the buffer and there are no empty elements below the line,
@@ -187,13 +165,12 @@ export class Buffer implements IBuffer {
             } else {
               // Add a blank line if there is no buffer left at the top to scroll to, or if there
               // are blank lines after the cursor
-              const fillCharData: CharData = [DEFAULT_ATTR, NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE];
-              this.lines.push(new this._bufferLineConstructor(newCols, fillCharData));
+              this.lines.push(new BufferLine(newCols, FILL_CHAR_DATA));
             }
           }
         }
-      } else { // (this._terminal.rows >= newRows)
-        for (let y = this._terminal.rows; y > newRows; y--) {
+      } else { // (this._rows >= newRows)
+        for (let y = this._rows; y > newRows; y--) {
           if (this.lines.length > newRows + this.ybase) {
             if (this.lines.length > this.ybase + this.y + 1) {
               // The line is a blank line below the cursor, remove it
@@ -233,8 +210,236 @@ export class Buffer implements IBuffer {
     }
 
     this.scrollBottom = newRows - 1;
+
+    if (this._isReflowEnabled) {
+      this._reflow(newCols, newRows);
+
+      // Trim the end of the line off if cols shrunk
+      if (this._cols > newCols) {
+        for (let i = 0; i < this.lines.length; i++) {
+          this.lines.get(i).resize(newCols, FILL_CHAR_DATA);
+        }
+      }
+    }
+
+    this._cols = newCols;
+    this._rows = newRows;
   }
 
+  private get _isReflowEnabled(): boolean {
+    return this._hasScrollback && !(this._terminal as any).isWinptyCompatEnabled;
+  }
+
+  private _reflow(newCols: number, newRows: number): void {
+    if (this._cols === newCols) {
+      return;
+    }
+
+    // Iterate through rows, ignore the last one as it cannot be wrapped
+    if (newCols > this._cols) {
+      this._reflowLarger(newCols, newRows);
+    } else {
+      this._reflowSmaller(newCols, newRows);
+    }
+  }
+
+  private _reflowLarger(newCols: number, newRows: number): void {
+    const toRemove: number[] = reflowLargerGetLinesToRemove(this.lines, newCols, this.ybase + this.y);
+    if (toRemove.length > 0) {
+      const newLayoutResult = reflowLargerCreateNewLayout(this.lines, toRemove);
+      reflowLargerApplyNewLayout(this.lines, newLayoutResult.layout);
+      this._reflowLargerAdjustViewport(newCols, newRows, newLayoutResult.countRemoved);
+    }
+  }
+
+  private _reflowLargerAdjustViewport(newCols: number, newRows: number, countRemoved: number): void {
+    // Adjust viewport based on number of items removed
+    let viewportAdjustments = countRemoved;
+    while (viewportAdjustments-- > 0) {
+      if (this.ybase === 0) {
+        if (this.y > 0) {
+          this.y--;
+        }
+        if (this.lines.length < newRows) {
+          // Add an extra row at the bottom of the viewport
+          this.lines.push(new BufferLine(newCols, FILL_CHAR_DATA));
+        }
+      } else {
+        if (this.ydisp === this.ybase) {
+          this.ydisp--;
+        }
+        this.ybase--;
+      }
+    }
+  }
+
+  private _reflowSmaller(newCols: number, newRows: number): void {
+    // Gather all BufferLines that need to be inserted into the Buffer here so that they can be
+    // batched up and only committed once
+    const toInsert = [];
+    let countToInsert = 0;
+    // Go backwards as many lines may be trimmed and this will avoid considering them
+    for (let y = this.lines.length - 1; y >= 0; y--) {
+      // Check whether this line is a problem
+      let nextLine = this.lines.get(y) as BufferLine;
+      if (!nextLine.isWrapped && nextLine.getTrimmedLength() <= newCols) {
+        continue;
+      }
+
+      // Gather wrapped lines and adjust y to be the starting line
+      const wrappedLines: BufferLine[] = [nextLine];
+      while (nextLine.isWrapped && y > 0) {
+        nextLine = this.lines.get(--y) as BufferLine;
+        wrappedLines.unshift(nextLine);
+      }
+
+      // If these lines contain the cursor don't touch them, the program will handle fixing up
+      // wrapped lines with the cursor
+      const absoluteY = this.ybase + this.y;
+      if (absoluteY >= y && absoluteY < y + wrappedLines.length) {
+        continue;
+      }
+
+      const lastLineLength = wrappedLines[wrappedLines.length - 1].getTrimmedLength();
+      const destLineLengths = reflowSmallerGetNewLineLengths(wrappedLines, this._cols, newCols);
+      const linesToAdd = destLineLengths.length - wrappedLines.length;
+      let trimmedLines: number;
+      if (this.ybase === 0 && this.y !== this.lines.length - 1) {
+        // If the top section of the buffer is not yet filled
+        trimmedLines = Math.max(0, this.y - this.lines.maxLength + linesToAdd);
+      } else {
+        trimmedLines = Math.max(0, this.lines.length - this.lines.maxLength + linesToAdd);
+      }
+
+      // Add the new lines
+      const newLines: BufferLine[] = [];
+      for (let i = 0; i < linesToAdd; i++) {
+        const newLine = this.getBlankLine(DEFAULT_ATTR, true) as BufferLine;
+        newLines.push(newLine);
+      }
+      if (newLines.length > 0) {
+        toInsert.push({
+          // countToInsert here gets the actual index, taking into account other inserted items.
+          // using this we can iterate through the list forwards
+          start: y + wrappedLines.length + countToInsert,
+          newLines
+        });
+        countToInsert += newLines.length;
+      }
+      wrappedLines.push(...newLines);
+
+      // Copy buffer data to new locations, this needs to happen backwards to do in-place
+      let destLineIndex = destLineLengths.length - 1; // Math.floor(cellsNeeded / newCols);
+      let destCol = destLineLengths[destLineIndex]; // cellsNeeded % newCols;
+      if (destCol === 0) {
+        destLineIndex--;
+        destCol = destLineLengths[destLineIndex];
+      }
+      let srcLineIndex = wrappedLines.length - linesToAdd - 1;
+      let srcCol = lastLineLength;
+      while (srcLineIndex >= 0) {
+        const cellsToCopy = Math.min(srcCol, destCol);
+        wrappedLines[destLineIndex].copyCellsFrom(wrappedLines[srcLineIndex], srcCol - cellsToCopy, destCol - cellsToCopy, cellsToCopy, true);
+        destCol -= cellsToCopy;
+        if (destCol === 0) {
+          destLineIndex--;
+          destCol = destLineLengths[destLineIndex];
+        }
+        srcCol -= cellsToCopy;
+        if (srcCol === 0) {
+          srcLineIndex--;
+          // TODO: srcCol shoudl take trimmed length into account
+          srcCol = wrappedLines[Math.max(srcLineIndex, 0)].getTrimmedLength(); // this._cols;
+        }
+      }
+
+      // Null out the end of the line ends if a wide character wrapped to the following line
+      for (let i = 0; i < wrappedLines.length; i++) {
+        if (destLineLengths[i] < newCols) {
+          wrappedLines[i].set(destLineLengths[i], FILL_CHAR_DATA);
+        }
+      }
+
+      // Adjust viewport as needed
+      let viewportAdjustments = linesToAdd - trimmedLines;
+      while (viewportAdjustments-- > 0) {
+        if (this.ybase === 0) {
+          if (this.y < newRows - 1) {
+            this.y++;
+            this.lines.pop();
+          } else {
+            this.ybase++;
+            this.ydisp++;
+          }
+        } else {
+          // Ensure ybase does not exceed its maximum value
+          if (this.ybase < Math.min(this.lines.maxLength, this.lines.length + countToInsert) - newRows) {
+            if (this.ybase === this.ydisp) {
+              this.ydisp++;
+            }
+            this.ybase++;
+          }
+        }
+      }
+    }
+
+    // Rearrange lines in the buffer if there are any insertions, this is done at the end rather
+    // than earlier so that it's a single O(n) pass through the buffer, instead of O(n^2) from many
+    // costly calls to CircularList.splice.
+    if (toInsert.length > 0) {
+      // Record buffer insert events and then play them back backwards so that the indexes are
+      // correct
+      const insertEvents: IInsertEvent[] = [];
+
+      // Record original lines so they don't get overridden when we rearrange the list
+      const originalLines: BufferLine[] = [];
+      for (let i = 0; i < this.lines.length; i++) {
+        originalLines.push(this.lines.get(i) as BufferLine);
+      }
+      const originalLinesLength = this.lines.length;
+
+      let originalLineIndex = originalLinesLength - 1;
+      let nextToInsertIndex = 0;
+      let nextToInsert = toInsert[nextToInsertIndex];
+      this.lines.length = Math.min(this.lines.maxLength, this.lines.length + countToInsert);
+      let countInsertedSoFar = 0;
+      for (let i = Math.min(this.lines.maxLength - 1, originalLinesLength + countToInsert - 1); i >= 0; i--) {
+        if (nextToInsert && nextToInsert.start > originalLineIndex + countInsertedSoFar) {
+          // Insert extra lines here, adjusting i as needed
+          for (let nextI = nextToInsert.newLines.length - 1; nextI >= 0; nextI--) {
+            this.lines.set(i--, nextToInsert.newLines[nextI]);
+          }
+          i++;
+
+          // Create insert events for later
+          insertEvents.push({
+            index: originalLineIndex + 1,
+            amount: nextToInsert.newLines.length
+          } as IInsertEvent);
+
+          countInsertedSoFar += nextToInsert.newLines.length;
+          nextToInsert = toInsert[++nextToInsertIndex];
+        } else {
+          this.lines.set(i, originalLines[originalLineIndex--]);
+        }
+      }
+
+      // Update markers
+      let insertCountEmitted = 0;
+      for (let i = insertEvents.length - 1; i >= 0; i--) {
+        insertEvents[i].index += insertCountEmitted;
+        this.lines.emit('insert', insertEvents[i]);
+        insertCountEmitted += insertEvents[i].amount;
+      }
+      const amountToTrim = Math.max(0, originalLinesLength + countToInsert - this.lines.maxLength);
+      if (amountToTrim > 0) {
+        this.lines.emitMayRemoveListeners('trim', amountToTrim);
+      }
+    }
+  }
+
+  // private _reflowSmallerGetLinesNeeded()
+
   /**
    * Translates a string index back to a BufferIndex.
    * To get the correct buffer position the string must start at `startCol` 0
@@ -248,14 +453,19 @@ export class Buffer implements IBuffer {
    * @param stringIndex index within the string
    * @param startCol column offset the string was retrieved from
    */
-  public stringIndexToBufferIndex(lineIndex: number, stringIndex: number): BufferIndex {
+  public stringIndexToBufferIndex(lineIndex: number, stringIndex: number, trimRight: boolean = false): BufferIndex {
     while (stringIndex) {
       const line = this.lines.get(lineIndex);
       if (!line) {
         return [-1, -1];
       }
-      for (let i = 0; i < line.length; ++i) {
-        stringIndex -= line.get(i)[CHAR_DATA_CHAR_INDEX].length;
+      const length = (trimRight) ? line.getTrimmedLength() : line.length;
+      for (let i = 0; i < length; ++i) {
+        if (line.get(i)[CHAR_DATA_WIDTH_INDEX]) {
+          // empty cells report a string length of 0, but get replaced
+          // with a whitespace in translateToString, thus replace with 1
+          stringIndex -= line.get(i)[CHAR_DATA_CHAR_INDEX].length || 1;
+        }
         if (stringIndex < 0) {
           return [lineIndex, i];
         }
@@ -311,7 +521,7 @@ export class Buffer implements IBuffer {
       i = 0;
     }
 
-    for (; i < this._terminal.cols; i += this._terminal.options.tabStopWidth) {
+    for (; i < this._cols; i += this._terminal.options.tabStopWidth) {
       this.tabs[i] = true;
     }
   }
@@ -325,7 +535,7 @@ export class Buffer implements IBuffer {
       x = this.x;
     }
     while (!this.tabs[--x] && x > 0);
-    return x >= this._terminal.cols ? this._terminal.cols - 1 : x < 0 ? 0 : x;
+    return x >= this._cols ? this._cols - 1 : x < 0 ? 0 : x;
   }
 
   /**
@@ -336,8 +546,8 @@ export class Buffer implements IBuffer {
     if (x === null || x === undefined) {
       x = this.x;
     }
-    while (!this.tabs[++x] && x < this._terminal.cols);
-    return x >= this._terminal.cols ? this._terminal.cols - 1 : x < 0 ? 0 : x;
+    while (!this.tabs[++x] && x < this._cols);
+    return x >= this._cols ? this._cols - 1 : x < 0 ? 0 : x;
   }
 
   public addMarker(y: number): Marker {
@@ -350,12 +560,27 @@ export class Buffer implements IBuffer {
         marker.dispose();
       }
     }));
+    marker.register(this.lines.addDisposableListener('insert', (event: IInsertEvent) => {
+      if (marker.line >= event.index) {
+        marker.line += event.amount;
+      }
+    }));
+    marker.register(this.lines.addDisposableListener('delete', (event: IDeleteEvent) => {
+      // Delete the marker if it's within the range
+      if (marker.line >= event.index && marker.line < event.index + event.amount) {
+        marker.dispose();
+      }
+
+      // Shift the marker if it's after the deleted range
+      if (marker.line > event.index) {
+        marker.line -= event.amount;
+      }
+    }));
     marker.register(marker.addDisposableListener('dispose', () => this._removeMarker(marker)));
     return marker;
   }
 
   private _removeMarker(marker: Marker): void {
-    // TODO: This could probably be optimized by relying on sort order and trimming the array using .length
     this.markers.splice(this.markers.indexOf(marker), 1);
   }
 

src/BufferLine.test.ts

@@ -9,6 +9,10 @@ import { NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE, DEFAULT_ATTR } from '.
 
 
 class TestBufferLine extends BufferLine {
+  public get combined(): {[index: number]: string} {
+    return this._combined;
+  }
+
   public toArray(): CharData[] {
     const result = [];
     for (let i = 0; i < this.length; ++i) {
@@ -141,63 +145,30 @@ describe('BufferLine', function(): void {
     });
     it('enlarge(true)', function(): void {
       const line = new TestBufferLine(5, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)], true);
+      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)]);
       chai.expect(line.toArray()).eql(Array(10).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
     });
     it('shrink(true) - should apply new size', function(): void {
       const line = new TestBufferLine(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(5, [1, 'a', 0, 'a'.charCodeAt(0)], true);
+      line.resize(5, [1, 'a', 0, 'a'.charCodeAt(0)]);
       chai.expect(line.toArray()).eql(Array(5).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
     });
-    it('shrink(false) - should not apply new size', function(): void {
-      const line = new TestBufferLine(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(5, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      chai.expect(line.toArray()).eql(Array(10).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-    });
-    it('shrink(false) + shrink(false) - should not apply new size', function(): void {
-      const line = new TestBufferLine(20, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(5, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      chai.expect(line.toArray()).eql(Array(20).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-    });
-    it('shrink(false) + enlarge(false) to smaller than before', function(): void {
-      const line = new TestBufferLine(20, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(15, [1, 'a', 0, 'a'.charCodeAt(0)]);
-      chai.expect(line.toArray()).eql(Array(20).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-    });
-    it('shrink(false) + enlarge(false) to bigger than before', function(): void {
-      const line = new TestBufferLine(20, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(25, [1, 'a', 0, 'a'.charCodeAt(0)]);
-      chai.expect(line.toArray()).eql(Array(25).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-    });
-    it('shrink(false) + resize shrink=true should enforce shrinking', function(): void {
-      const line = new TestBufferLine(20, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)], true);
-      chai.expect(line.toArray()).eql(Array(10).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-    });
-    it('enlarge from 0 length', function(): void {
-      const line = new TestBufferLine(0, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      chai.expect(line.toArray()).eql(Array(10).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-    });
     it('shrink to 0 length', function(): void {
       const line = new TestBufferLine(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(0, [1, 'a', 0, 'a'.charCodeAt(0)], true);
+      line.resize(0, [1, 'a', 0, 'a'.charCodeAt(0)]);
       chai.expect(line.toArray()).eql(Array(0).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
     });
-    it('shrink(false) to 0 and enlarge to different sizes', function(): void {
+    it('should remove combining data on replaced cells after shrinking then enlarging', () => {
       const line = new TestBufferLine(10, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      line.resize(0, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      chai.expect(line.toArray()).eql(Array(10).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-      line.resize(5, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      chai.expect(line.toArray()).eql(Array(10).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-      line.resize(7, [1, 'a', 0, 'a'.charCodeAt(0)], false);
-      chai.expect(line.toArray()).eql(Array(10).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
-      line.resize(7, [1, 'a', 0, 'a'.charCodeAt(0)], true);
-      chai.expect(line.toArray()).eql(Array(7).fill([1, 'a', 0, 'a'.charCodeAt(0)]));
+      line.set(2, [ null, '😁', 1, '😁'.charCodeAt(0) ]);
+      line.set(9, [ null, '😁', 1, '😁'.charCodeAt(0) ]);
+      chai.expect(line.translateToString()).eql('aa😁aaaaaa😁');
+      chai.expect(Object.keys(line.combined).length).eql(2);
+      line.resize(5, [1, 'a', 0, 'a'.charCodeAt(0)]);
+      chai.expect(line.translateToString()).eql('aa😁aa');
+      line.resize(10, [1, 'a', 0, 'a'.charCodeAt(0)]);
+      chai.expect(line.translateToString()).eql('aa😁aaaaaaa');
+      chai.expect(Object.keys(line.combined).length).eql(1);
     });
   });
   describe('getTrimLength', function(): void {

src/BufferLine.ts

@@ -3,133 +3,8 @@
  * @license MIT
  */
 import { CharData, IBufferLine } from './Types';
-import { NULL_CELL_CODE, NULL_CELL_WIDTH, NULL_CELL_CHAR, CHAR_DATA_CHAR_INDEX, CHAR_DATA_WIDTH_INDEX, WHITESPACE_CELL_CHAR } from './Buffer';
+import { NULL_CELL_CODE, NULL_CELL_WIDTH, NULL_CELL_CHAR, WHITESPACE_CELL_CHAR } from './Buffer';
 
-/**
- * Class representing a terminal line.
- *
- * @deprecated to be removed with one of the next releases
- */
-export class BufferLineJSArray implements IBufferLine {
-  protected _data: CharData[];
-  public isWrapped = false;
-  public length: number;
-
-  constructor(cols: number, fillCharData?: CharData, isWrapped?: boolean) {
-    this._data = [];
-    if (!fillCharData) {
-      fillCharData = [0, NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE];
-    }
-    for (let i = 0; i < cols; i++) {
-      this._push(fillCharData);  // Note: the ctor ch is not cloned (resembles old behavior)
-    }
-    if (isWrapped) {
-      this.isWrapped = true;
-    }
-    this.length = this._data.length;
-  }
-
-  private _pop(): CharData | undefined  {
-    const data = this._data.pop();
-    this.length = this._data.length;
-    return data;
-  }
-
-  private _push(data: CharData): void {
-    this._data.push(data);
-    this.length = this._data.length;
-  }
-
-  private _splice(start: number, deleteCount: number, ...items: CharData[]): CharData[] {
-    const removed = this._data.splice(start, deleteCount, ...items);
-    this.length = this._data.length;
-    return removed;
-  }
-
-  public get(index: number): CharData {
-    return this._data[index];
-  }
-
-  public set(index: number, data: CharData): void {
-    this._data[index] = data;
-  }
-
-  /** insert n cells ch at pos, right cells are lost (stable length)  */
-  public insertCells(pos: number, n: number, ch: CharData): void {
-    while (n--) {
-      this._splice(pos, 0, ch);
-      this._pop();
-    }
-  }
-
-  /** delete n cells at pos, right side is filled with fill (stable length) */
-  public deleteCells(pos: number, n: number, fillCharData: CharData): void {
-    while (n--) {
-      this._splice(pos, 1);
-      this._push(fillCharData);
-    }
-  }
-
-  /** replace cells from pos to pos + n - 1 with fill */
-  public replaceCells(start: number, end: number, fillCharData: CharData): void {
-    while (start < end  && start < this.length) {
-      this.set(start++, fillCharData);  // Note: fill is not cloned (resembles old behavior)
-    }
-  }
-
-  /** resize line to cols filling new cells with fill */
-  public resize(cols: number, fillCharData: CharData, shrink: boolean = false): void {
-    while (this._data.length < cols) {
-      this._data.push(fillCharData);
-    }
-    if (shrink) {
-      while (this._data.length > cols) {
-        this._data.pop();
-      }
-    }
-    this.length = this._data.length;
-  }
-
-  public fill(fillCharData: CharData): void {
-    for (let i = 0; i < this.length; ++i) {
-      this.set(i, fillCharData);
-    }
-  }
-
-  public copyFrom(line: BufferLineJSArray): void {
-    this._data = line._data.slice(0);
-    this.length = line.length;
-    this.isWrapped = line.isWrapped;
-  }
-
-  public clone(): IBufferLine {
-    const newLine = new BufferLineJSArray(0);
-    newLine.copyFrom(this);
-    return newLine;
-  }
-
-  public getTrimmedLength(): number {
-    for (let i = this.length - 1; i >= 0; --i) {
-      const ch = this.get(i);
-      if (ch[CHAR_DATA_CHAR_INDEX] !== '') {
-        return i + ch[CHAR_DATA_WIDTH_INDEX];
-      }
-    }
-    return 0;
-  }
-
-  public translateToString(trimRight: boolean = false, startCol: number = 0, endCol: number = this.length): string {
-    if (trimRight) {
-      endCol = Math.min(endCol, this.getTrimmedLength());
-    }
-    let result = '';
-    while (startCol < endCol) {
-      result += this.get(startCol)[CHAR_DATA_CHAR_INDEX] || WHITESPACE_CELL_CHAR;
-      startCol += this.get(startCol)[CHAR_DATA_WIDTH_INDEX] || 1;
-    }
-    return result;
-  }
-}
 
 /** typed array slots taken by one cell */
 const CELL_SIZE = 3;
@@ -179,6 +54,10 @@ export class BufferLine implements IBufferLine {
     ];
   }
 
+  public getWidth(index: number): number {
+    return this._data[index * CELL_SIZE + Cell.WIDTH];
+  }
+
   public set(index: number, value: CharData): void {
     this._data[index * CELL_SIZE + Cell.FLAGS] = value[0];
     if (value[1].length > 1) {
@@ -228,8 +107,8 @@ export class BufferLine implements IBufferLine {
     }
   }
 
-  public resize(cols: number, fillCharData: CharData, shrink: boolean = false): void {
-    if (cols === this.length || (!shrink && cols < this.length)) {
+  public resize(cols: number, fillCharData: CharData): void {
+    if (cols === this.length) {
       return;
     }
     if (cols > this.length) {
@@ -245,13 +124,22 @@ export class BufferLine implements IBufferLine {
       for (let i = this.length; i < cols; ++i) {
         this.set(i, fillCharData);
       }
-    } else if (shrink) {
+    } else {
       if (cols) {
         const data = new Uint32Array(cols * CELL_SIZE);
         data.set(this._data.subarray(0, cols * CELL_SIZE));
         this._data = data;
+        // Remove any cut off combined data
+        const keys = Object.keys(this._combined);
+        for (let i = 0; i < keys.length; i++) {
+          const key = parseInt(keys[i], 10);
+          if (key >= cols) {
+            delete this._combined[key];
+          }
+        }
       } else {
         this._data = null;
+        this._combined = {};
       }
     }
     this.length = cols;
@@ -304,6 +192,32 @@ export class BufferLine implements IBufferLine {
     return 0;
   }
 
+  public copyCellsFrom(src: BufferLine, srcCol: number, destCol: number, length: number, applyInReverse: boolean): void {
+    const srcData = src._data;
+    if (applyInReverse) {
+      for (let cell = length - 1; cell >= 0; cell--) {
+        for (let i = 0; i < CELL_SIZE; i++) {
+          this._data[(destCol + cell) * CELL_SIZE + i] = srcData[(srcCol + cell) * CELL_SIZE + i];
+        }
+      }
+    } else {
+      for (let cell = 0; cell < length; cell++) {
+        for (let i = 0; i < CELL_SIZE; i++) {
+          this._data[(destCol + cell) * CELL_SIZE + i] = srcData[(srcCol + cell) * CELL_SIZE + i];
+        }
+      }
+    }
+
+    // Move any combined data over as needed
+    const srcCombinedKeys = Object.keys(src._combined);
+    for (let i = 0; i < srcCombinedKeys.length; i++) {
+      const key = parseInt(srcCombinedKeys[i], 10);
+      if (key >= srcCol) {
+        this._combined[key - srcCol + destCol] = src._combined[key];
+      }
+    }
+  }
+
   public translateToString(trimRight: boolean = false, startCol: number = 0, endCol: number = this.length): string {
     if (trimRight) {
       endCol = Math.min(endCol, this.getTrimmedLength());

src/BufferReflow.test.ts

@@ -0,0 +1,93 @@
+/**
+ * Copyright (c) 2019 The xterm.js authors. All rights reserved.
+ * @license MIT
+ */
+import { assert } from 'chai';
+import { BufferLine } from './BufferLine';
+import { reflowSmallerGetNewLineLengths } from './BufferReflow';
+import { NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE } from './Buffer';
+
+describe('BufferReflow', () => {
+  describe('reflowSmallerGetNewLineLengths', () => {
+    it('should return correct line lengths for a small line with wide characters', () => {
+      const line = new BufferLine(4);
+      line.set(0, [null, '汉', 2, '汉'.charCodeAt(0)]);
+      line.set(1, [null, '', 0, undefined]);
+      line.set(2, [null, '语', 2, '语'.charCodeAt(0)]);
+      line.set(3, [null, '', 0, undefined]);
+      assert.equal(line.translateToString(true), '汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 4, 3), [2, 2], 'line: 汉, 语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 4, 2), [2, 2], 'line: 汉, 语');
+    });
+    it('should return correct line lengths for a large line with wide characters', () => {
+      const line = new BufferLine(12);
+      for (let i = 0; i < 12; i += 4) {
+        line.set(i, [null, '汉', 2, '汉'.charCodeAt(0)]);
+        line.set(i + 2, [null, '语', 2, '语'.charCodeAt(0)]);
+      }
+      for (let i = 1; i < 12; i += 2) {
+        line.set(i, [null, '', 0, undefined]);
+        line.set(i, [null, '', 0, undefined]);
+      }
+      assert.equal(line.translateToString(), '汉语汉语汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 11), [10, 2], 'line: 汉语汉语汉, 语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 10), [10, 2], 'line: 汉语汉语汉, 语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 9), [8, 4], 'line: 汉语汉语, 汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 8), [8, 4], 'line: 汉语汉语, 汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 7), [6, 6], 'line: 汉语汉, 语汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 6), [6, 6], 'line: 汉语汉, 语汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 5), [4, 4, 4], 'line: 汉语, 汉语, 汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 4), [4, 4, 4], 'line: 汉语, 汉语, 汉语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 3), [2, 2, 2, 2, 2, 2], 'line: 汉, 语, 汉, 语, 汉, 语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 12, 2), [2, 2, 2, 2, 2, 2], 'line: 汉, 语, 汉, 语, 汉, 语');
+    });
+    it('should return correct line lengths for a string with wide and single characters', () => {
+      const line = new BufferLine(6);
+      line.set(0, [null, 'a', 1, 'a'.charCodeAt(0)]);
+      line.set(1, [null, '汉', 2, '汉'.charCodeAt(0)]);
+      line.set(2, [null, '', 0, undefined]);
+      line.set(3, [null, '语', 2, '语'.charCodeAt(0)]);
+      line.set(4, [null, '', 0, undefined]);
+      line.set(5, [null, 'b', 1, 'b'.charCodeAt(0)]);
+      assert.equal(line.translateToString(), 'a汉语b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 6, 5), [5, 1], 'line: a汉语b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 6, 4), [3, 3], 'line: a汉, 语b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 6, 3), [3, 3], 'line: a汉, 语b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 6, 2), [1, 2, 2, 1], 'line: a, 汉, 语, b');
+    });
+    it('should return correct line lengths for a wrapped line with wide and single characters', () => {
+      const line1 = new BufferLine(6);
+      line1.set(0, [null, 'a', 1, 'a'.charCodeAt(0)]);
+      line1.set(1, [null, '汉', 2, '汉'.charCodeAt(0)]);
+      line1.set(2, [null, '', 0, undefined]);
+      line1.set(3, [null, '语', 2, '语'.charCodeAt(0)]);
+      line1.set(4, [null, '', 0, undefined]);
+      line1.set(5, [null, 'b', 1, 'b'.charCodeAt(0)]);
+      const line2 = new BufferLine(6, undefined, true);
+      line2.set(0, [null, 'a', 1, 'a'.charCodeAt(0)]);
+      line2.set(1, [null, '汉', 2, '汉'.charCodeAt(0)]);
+      line2.set(2, [null, '', 0, undefined]);
+      line2.set(3, [null, '语', 2, '语'.charCodeAt(0)]);
+      line2.set(4, [null, '', 0, undefined]);
+      line2.set(5, [null, 'b', 1, 'b'.charCodeAt(0)]);
+      assert.equal(line1.translateToString(), 'a汉语b');
+      assert.equal(line2.translateToString(), 'a汉语b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line1, line2], 6, 5), [5, 4, 3], 'lines: a汉语, ba汉, 语b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line1, line2], 6, 4), [3, 4, 4, 1], 'lines: a汉, 语ba, 汉语, b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line1, line2], 6, 3), [3, 3, 3, 3], 'lines: a汉, 语b, a汉, 语b');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line1, line2], 6, 2), [1, 2, 2, 2, 2, 2, 1], 'lines: a, 汉, 语, ba, 汉, 语, b');
+    });
+    it('should work on lines ending in null space', () => {
+      const line = new BufferLine(5);
+      line.set(0, [null, '汉', 2, '汉'.charCodeAt(0)]);
+      line.set(1, [null, '', 0, undefined]);
+      line.set(2, [null, '语', 2, '语'.charCodeAt(0)]);
+      line.set(3, [null, '', 0, undefined]);
+      line.set(4, [null, NULL_CELL_CHAR, NULL_CELL_WIDTH, NULL_CELL_CODE]);
+      assert.equal(line.translateToString(true), '汉语');
+      assert.equal(line.translateToString(false), '汉语 ');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 4, 3), [2, 2], 'line: 汉, 语');
+      assert.deepEqual(reflowSmallerGetNewLineLengths([line], 4, 2), [2, 2], 'line: 汉, 语');
+    });
+  });
+});

src/BufferReflow.ts

@@ -0,0 +1,205 @@
+/**
+ * Copyright (c) 2019 The xterm.js authors. All rights reserved.
+ * @license MIT
+ */
+
+import { FILL_CHAR_DATA } from './Buffer';
+import { BufferLine } from './BufferLine';
+import { CircularList, IDeleteEvent } from './common/CircularList';
+import { IBufferLine } from './Types';
+
+export interface INewLayoutResult {
+  layout: number[];
+  countRemoved: number;
+}
+
+/**
+ * Evaluates and returns indexes to be removed after a reflow larger occurs. Lines will be removed
+ * when a wrapped line unwraps.
+ * @param lines The buffer lines.
+ * @param newCols The columns after resize.
+ */
+export function reflowLargerGetLinesToRemove(lines: CircularList<IBufferLine>, newCols: number, bufferAbsoluteY: number): number[] {
+  // Gather all BufferLines that need to be removed from the Buffer here so that they can be
+  // batched up and only committed once
+  const toRemove: number[] = [];
+
+  for (let y = 0; y < lines.length - 1; y++) {
+    // Check if this row is wrapped
+    let i = y;
+    let nextLine = lines.get(++i) as BufferLine;
+    if (!nextLine.isWrapped) {
+      continue;
+    }
+
+    // Check how many lines it's wrapped for
+    const wrappedLines: BufferLine[] = [lines.get(y) as BufferLine];
+    while (i < lines.length && nextLine.isWrapped) {
+      wrappedLines.push(nextLine);
+      nextLine = lines.get(++i) as BufferLine;
+    }
+
+    // If these lines contain the cursor don't touch them, the program will handle fixing up wrapped
+    // lines with the cursor
+    if (bufferAbsoluteY >= y && bufferAbsoluteY < i) {
+      y += wrappedLines.length - 1;
+      continue;
+    }
+
+    // Copy buffer data to new locations
+    let destLineIndex = 0;
+    let destCol = wrappedLines[destLineIndex].getTrimmedLength();
+    let srcLineIndex = 1;
+    let srcCol = 0;
+    while (srcLineIndex < wrappedLines.length) {
+      const srcTrimmedTineLength = wrappedLines[srcLineIndex].getTrimmedLength();
+      const srcRemainingCells = srcTrimmedTineLength - srcCol;
+      const destRemainingCells = newCols - destCol;
+      const cellsToCopy = Math.min(srcRemainingCells, destRemainingCells);
+
+      wrappedLines[destLineIndex].copyCellsFrom(wrappedLines[srcLineIndex], srcCol, destCol, cellsToCopy, false);
+
+      destCol += cellsToCopy;
+      if (destCol === newCols) {
+        destLineIndex++;
+        destCol = 0;
+      }
+      srcCol += cellsToCopy;
+      if (srcCol === srcTrimmedTineLength) {
+        srcLineIndex++;
+        srcCol = 0;
+      }
+
+      // Make sure the last cell isn't wide, if it is copy it to the current dest
+      if (destCol === 0 && destLineIndex !== 0) {
+        if (wrappedLines[destLineIndex - 1].getWidth(newCols - 1) === 2) {
+          wrappedLines[destLineIndex].copyCellsFrom(wrappedLines[destLineIndex - 1], newCols - 1, destCol++, 1, false);
+          // Null out the end of the last row
+          wrappedLines[destLineIndex - 1].set(newCols - 1, FILL_CHAR_DATA);
+        }
+      }
+    }
+
+    // Clear out remaining cells or fragments could remain;
+    wrappedLines[destLineIndex].replaceCells(destCol, newCols, FILL_CHAR_DATA);
+
+    // Work backwards and remove any rows at the end that only contain null cells
+    let countToRemove = 0;
+    for (let i = wrappedLines.length - 1; i > 0; i--) {
+      if (i > destLineIndex || wrappedLines[i].getTrimmedLength() === 0) {
+        countToRemove++;
+      } else {
+        break;
+      }
+    }
+
+    if (countToRemove > 0) {
+      toRemove.push(y + wrappedLines.length - countToRemove); // index
+      toRemove.push(countToRemove);
+    }
+
+    y += wrappedLines.length - 1;
+  }
+  return toRemove;
+}
+
+/**
+ * Creates and return the new layout for lines given an array of indexes to be removed.
+ * @param lines The buffer lines.
+ * @param toRemove The indexes to remove.
+ */
+export function reflowLargerCreateNewLayout(lines: CircularList<IBufferLine>, toRemove: number[]): INewLayoutResult {
+  const layout: number[] = [];
+  // First iterate through the list and get the actual indexes to use for rows
+  let nextToRemoveIndex = 0;
+  let nextToRemoveStart = toRemove[nextToRemoveIndex];
+  let countRemovedSoFar = 0;
+  for (let i = 0; i < lines.length; i++) {
+    if (nextToRemoveStart === i) {
+      const countToRemove = toRemove[++nextToRemoveIndex];
+
+      // Tell markers that there was a deletion
+      lines.emit('delete', {
+        index: i - countRemovedSoFar,
+        amount: countToRemove
+      } as IDeleteEvent);
+
+      i += countToRemove - 1;
+      countRemovedSoFar += countToRemove;
+      nextToRemoveStart = toRemove[++nextToRemoveIndex];
+    } else {
+      layout.push(i);
+    }
+  }
+  return {
+    layout,
+    countRemoved: countRemovedSoFar
+  };
+}
+
+/**
+ * Applies a new layout to the buffer. This essentially does the same as many splice calls but it's
+ * done all at once in a single iteration through the list since splice is very expensive.
+ * @param lines The buffer lines.
+ * @param newLayout The new layout to apply.
+ */
+export function reflowLargerApplyNewLayout(lines: CircularList<IBufferLine>, newLayout: number[]): void {
+  // Record original lines so they don't get overridden when we rearrange the list
+  const newLayoutLines: BufferLine[] = [];
+  for (let i = 0; i < newLayout.length; i++) {
+    newLayoutLines.push(lines.get(newLayout[i]) as BufferLine);
+  }
+
+  // Rearrange the list
+  for (let i = 0; i < newLayoutLines.length; i++) {
+    lines.set(i, newLayoutLines[i]);
+  }
+  lines.length = newLayout.length;
+}
+
+/**
+ * Gets the new line lengths for a given wrapped line. The purpose of this function it to pre-
+ * compute the wrapping points since wide characters may need to be wrapped onto the following line.
+ * This function will return an array of numbers of where each line wraps to, the resulting array
+ * will only contain the values `newCols` (when the line does not end with a wide character) and
+ * `newCols - 1` (when the line does end with a wide character), except for the last value which
+ * will contain the remaining items to fill the line.
+ *
+ * Calling this with a `newCols` value of `1` will lock up.
+ *
+ * @param wrappedLines The wrapped lines to evaluate.
+ * @param oldCols The columns before resize.
+ * @param newCols The columns after resize.
+ */
+export function reflowSmallerGetNewLineLengths(wrappedLines: BufferLine[], oldCols: number, newCols: number): number[] {
+  const newLineLengths: number[] = [];
+  const cellsNeeded = wrappedLines.map(l => l.getTrimmedLength()).reduce((p, c) => p + c);
+
+  // Use srcCol and srcLine to find the new wrapping point, use that to get the cellsAvailable and
+  // linesNeeded
+  let srcCol = 0;
+  let srcLine = 0;
+  let cellsAvailable = 0;
+  while (cellsAvailable < cellsNeeded) {
+    if (cellsNeeded - cellsAvailable < newCols) {
+      // Add the final line and exit the loop
+      newLineLengths.push(cellsNeeded - cellsAvailable);
+      break;
+    }
+    srcCol += newCols;
+    const oldTrimmedLength = wrappedLines[srcLine].getTrimmedLength();
+    if (srcCol > oldTrimmedLength) {
+      srcCol -= oldTrimmedLength;
+      srcLine++;
+    }
+    const endsWithWide = wrappedLines[srcLine].getWidth(srcCol - 1) === 2;
+    if (endsWithWide) {
+      srcCol--;
+    }
+    const lineLength = endsWithWide ? newCols - 1 : newCols;
+    newLineLengths.push(lineLength);
+    cellsAvailable += lineLength;
+  }
+
+  return newLineLengths;
+}

src/BufferSet.test.ts

@@ -15,8 +15,8 @@ describe('BufferSet', () => {
 
   beforeEach(() => {
     terminal = new MockTerminal();
-    terminal.cols = 80;
-    terminal.rows = 24;
+    (terminal as any).cols = 80;
+    (terminal as any).rows = 24;
     terminal.options.scrollback = 1000;
     bufferSet = new BufferSet(terminal);
   });

src/CompositionHelper.ts

@@ -111,17 +111,17 @@ export class CompositionHelper {
   /**
    * Finalizes the composition, resuming regular input actions. This is called when a composition
    * is ending.
-   * @param waitForPropogation Whether to wait for events to propogate before sending
+   * @param waitForPropagation Whether to wait for events to propagate before sending
    *   the input. This should be false if a non-composition keystroke is entered before the
-   *   compositionend event is triggered, such as enter, so that the composition is send before
+   *   compositionend event is triggered, such as enter, so that the composition is sent before
    *   the command is executed.
    */
-  private _finalizeComposition(waitForPropogation: boolean): void {
+  private _finalizeComposition(waitForPropagation: boolean): void {
     this._compositionView.classList.remove('active');
     this._isComposing = false;
     this._clearTextareaPosition();
 
-    if (!waitForPropogation) {
+    if (!waitForPropagation) {
       // Cancel any delayed composition send requests and send the input immediately.
       this._isSendingComposition = false;
       const input = this._textarea.value.substring(this._compositionPosition.start, this._compositionPosition.end);
@@ -136,8 +136,8 @@ export class CompositionHelper {
 
       // Since composition* events happen before the changes take place in the textarea on most
       // browsers, use a setTimeout with 0ms time to allow the native compositionend event to
-      // complete. This ensures the correct character is retrieved, this solution was used
-      // because:
+      // complete. This ensures the correct character is retrieved.
+      // This solution was used because:
       // - The compositionend event's data property is unreliable, at least on Chromium
       // - The last compositionupdate event's data property does not always accurately describe
       //   the character, a counter example being Korean where an ending consonsant can move to
@@ -203,6 +203,8 @@ export class CompositionHelper {
       this._compositionView.style.top = cursorTop + 'px';
       this._compositionView.style.height = cellHeight + 'px';
       this._compositionView.style.lineHeight = cellHeight + 'px';
+      this._compositionView.style.fontFamily = this._terminal.options.fontFamily;
+      this._compositionView.style.fontSize = this._terminal.options.fontSize + 'px';
       // Sync the textarea to the exact position of the composition view so the IME knows where the
       // text is.
       const compositionViewBounds = this._compositionView.getBoundingClientRect();

src/EscapeSequenceParser.test.ts

@@ -6,6 +6,7 @@
 import { ParserState, IDcsHandler, IParsingState } from './Types';
 import { EscapeSequenceParser, TransitionTable, VT500_TRANSITION_TABLE } from './EscapeSequenceParser';
 import * as chai from 'chai';
+import { StringToUtf32, stringFromCodePoint } from './core/input/TextDecoder';
 
 function r(a: number, b: number): string[] {
   let c = b - a;
@@ -50,8 +51,12 @@ const testTerminal: any = {
   compare: function (value: any): void {
     chai.expect(this.calls.slice()).eql(value); // weird bug w'o slicing here
   },
-  print: function (data: string, start: number, end: number): void {
-    this.calls.push(['print', data.substring(start, end)]);
+  print: function (data: Uint32Array, start: number, end: number): void {
+    let s = '';
+    for (let i = start; i < end; ++i) {
+      s += stringFromCodePoint(data[i]);
+    }
+    this.calls.push(['print', s]);
   },
   actionOSC: function (s: string): void {
     this.calls.push(['osc', s]);
@@ -68,8 +73,12 @@ const testTerminal: any = {
   actionDCSHook: function (collect: string, params: number[], flag: string): void {
     this.calls.push(['dcs hook', collect, params, flag]);
   },
-  actionDCSPrint: function (data: string, start: number, end: number): void {
-    this.calls.push(['dcs put', data.substring(start, end)]);
+  actionDCSPrint: function (data: Uint32Array, start: number, end: number): void {
+    let s = '';
+    for (let i = start; i < end; ++i) {
+      s += stringFromCodePoint(data[i]);
+    }
+    this.calls.push(['dcs put', s]);
   },
   actionDCSUnhook: function (): void {
     this.calls.push(['dcs unhook']);
@@ -81,7 +90,7 @@ class DcsTest implements IDcsHandler {
   hook(collect: string, params: number[], flag: number): void {
     testTerminal.actionDCSHook(collect, params, String.fromCharCode(flag));
   }
-  put(data: string, start: number, end: number): void {
+  put(data: Uint32Array, start: number, end: number): void {
     testTerminal.actionDCSPrint(data, start, end);
   }
   unhook(): void {
@@ -155,6 +164,12 @@ interface IRun {
   parser: TestEscapeSequenceParser;
 }
 
+// translate string based parse calls into typed array based
+function parse(parser: TestEscapeSequenceParser, data: string): void {
+  const container = new Uint32Array(data.length);
+  const decoder = new StringToUtf32();
+  parser.parse(container, decoder.decode(data, container));
+}
 
 describe('EscapeSequenceParser', function (): void {
   let parser: TestEscapeSequenceParser | null = null;
@@ -205,12 +220,12 @@ describe('EscapeSequenceParser', function (): void {
       it('state GROUND execute action', function (): void {
         parser.reset();
         testTerminal.clear();
-        const exes = r(0x00, 0x18);
-        exes.concat(['\x19']);
-        exes.concat(r(0x1c, 0x20));
+        let exes = r(0x00, 0x18);
+        exes = exes.concat(['\x19']);
+        exes = exes.concat(r(0x1c, 0x20));
         for (let i = 0; i < exes.length; ++i) {
           parser.currentState = ParserState.GROUND;
-          parser.parse(exes[i]);
+          parse(parser, exes[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([['exe', exes[i]]]);
           parser.reset();
@@ -223,7 +238,7 @@ describe('EscapeSequenceParser', function (): void {
         const printables = r(0x20, 0x7f); // NOTE: DEL excluded
         for (let i = 0; i < printables.length; ++i) {
           parser.currentState = ParserState.GROUND;
-          parser.parse(printables[i]);
+          parse(parser, printables[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([['print', printables[i]]]);
           parser.reset();
@@ -245,13 +260,13 @@ describe('EscapeSequenceParser', function (): void {
         for (state in states) {
           for (let i = 0; i < exes.length; ++i) {
             parser.currentState = state;
-            parser.parse(exes[i]);
+            parse(parser, exes[i]);
             chai.expect(parser.currentState).equal(ParserState.GROUND);
             testTerminal.compare((state in exceptions ? exceptions[state][exes[i]] : 0) || [['exe', exes[i]]]);
             parser.reset();
             testTerminal.clear();
           }
-          parser.parse('\x9c');
+          parse(parser, '\x9c');
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([]);
           parser.reset();
@@ -265,7 +280,7 @@ describe('EscapeSequenceParser', function (): void {
           parser.osc = '#';
           parser.params = [23];
           parser.collect = '#';
-          parser.parse('\x1b');
+          parse(parser, '\x1b');
           chai.expect(parser.currentState).equal(ParserState.ESCAPE);
           chai.expect(parser.osc).equal('');
           chai.expect(parser.params).eql([0]);
@@ -276,12 +291,12 @@ describe('EscapeSequenceParser', function (): void {
       it('state ESCAPE execute rules', function (): void {
         parser.reset();
         testTerminal.clear();
-        const exes = r(0x00, 0x18);
-        exes.concat(['\x19']);
-        exes.concat(r(0x1c, 0x20));
+        let exes = r(0x00, 0x18);
+        exes = exes.concat(['\x19']);
+        exes = exes.concat(r(0x1c, 0x20));
         for (let i = 0; i < exes.length; ++i) {
           parser.currentState = ParserState.ESCAPE;
-          parser.parse(exes[i]);
+          parse(parser, exes[i]);
           chai.expect(parser.currentState).equal(ParserState.ESCAPE);
           testTerminal.compare([['exe', exes[i]]]);
           parser.reset();
@@ -292,7 +307,7 @@ describe('EscapeSequenceParser', function (): void {
         parser.reset();
         testTerminal.clear();
         parser.currentState = ParserState.ESCAPE;
-        parser.parse('\x7f');
+        parse(parser, '\x7f');
         chai.expect(parser.currentState).equal(ParserState.ESCAPE);
         testTerminal.compare([]);
         parser.reset();
@@ -301,13 +316,13 @@ describe('EscapeSequenceParser', function (): void {
       it('trans ESCAPE --> GROUND with ecs_dispatch action', function (): void {
         parser.reset();
         testTerminal.clear();
-        const dispatches = r(0x30, 0x50);
-        dispatches.concat(r(0x51, 0x58));
-        dispatches.concat(['\x59', '\x5a', '\x5c']);
-        dispatches.concat(r(0x60, 0x7f));
+        let dispatches = r(0x30, 0x50);
+        dispatches = dispatches.concat(r(0x51, 0x58));
+        dispatches = dispatches.concat(['\x59', '\x5a']); // excluded \x5c
+        dispatches = dispatches.concat(r(0x60, 0x7f));
         for (let i = 0; i < dispatches.length; ++i) {
           parser.currentState = ParserState.ESCAPE;
-          parser.parse(dispatches[i]);
+          parse(parser, dispatches[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([['esc', '', dispatches[i]]]);
           parser.reset();
@@ -319,7 +334,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.ESCAPE;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.ESCAPE_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -328,12 +343,12 @@ describe('EscapeSequenceParser', function (): void {
       it('state ESCAPE_INTERMEDIATE execute rules', function (): void {
         parser.reset();
         testTerminal.clear();
-        const exes = r(0x00, 0x18);
-        exes.concat(['\x19']);
-        exes.concat(r(0x1c, 0x20));
+        let exes = r(0x00, 0x18);
+        exes = exes.concat(['\x19']);
+        exes = exes.concat(r(0x1c, 0x20));
         for (let i = 0; i < exes.length; ++i) {
           parser.currentState = ParserState.ESCAPE_INTERMEDIATE;
-          parser.parse(exes[i]);
+          parse(parser, exes[i]);
           chai.expect(parser.currentState).equal(ParserState.ESCAPE_INTERMEDIATE);
           testTerminal.compare([['exe', exes[i]]]);
           parser.reset();
@@ -344,7 +359,7 @@ describe('EscapeSequenceParser', function (): void {
         parser.reset();
         testTerminal.clear();
         parser.currentState = ParserState.ESCAPE_INTERMEDIATE;
-        parser.parse('\x7f');
+        parse(parser, '\x7f');
         chai.expect(parser.currentState).equal(ParserState.ESCAPE_INTERMEDIATE);
         testTerminal.compare([]);
         parser.reset();
@@ -355,7 +370,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.ESCAPE_INTERMEDIATE;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.ESCAPE_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -367,7 +382,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x30, 0x7f);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.ESCAPE_INTERMEDIATE;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           // '\x5c' --> ESC + \ (7bit ST) parser does not expose this as it already got handled
           testTerminal.compare((collect[i] === '\x5c') ? [] : [['esc', '', collect[i]]]);
@@ -382,7 +397,7 @@ describe('EscapeSequenceParser', function (): void {
         parser.osc = '#';
         parser.params = [123];
         parser.collect = '#';
-        parser.parse('[');
+        parse(parser, '[');
         chai.expect(parser.currentState).equal(ParserState.CSI_ENTRY);
         chai.expect(parser.osc).equal('');
         chai.expect(parser.params).eql([0]);
@@ -394,7 +409,7 @@ describe('EscapeSequenceParser', function (): void {
           parser.osc = '#';
           parser.params = [123];
           parser.collect = '#';
-          parser.parse('\x9b');
+          parse(parser, '\x9b');
           chai.expect(parser.currentState).equal(ParserState.CSI_ENTRY);
           chai.expect(parser.osc).equal('');
           chai.expect(parser.params).eql([0]);
@@ -405,12 +420,12 @@ describe('EscapeSequenceParser', function (): void {
       it('state CSI_ENTRY execute rules', function (): void {
         parser.reset();
         testTerminal.clear();
-        const exes = r(0x00, 0x18);
-        exes.concat(['\x19']);
-        exes.concat(r(0x1c, 0x20));
+        let exes = r(0x00, 0x18);
+        exes = exes.concat(['\x19']);
+        exes = exes.concat(r(0x1c, 0x20));
         for (let i = 0; i < exes.length; ++i) {
           parser.currentState = ParserState.CSI_ENTRY;
-          parser.parse(exes[i]);
+          parse(parser, exes[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_ENTRY);
           testTerminal.compare([['exe', exes[i]]]);
           parser.reset();
@@ -421,7 +436,7 @@ describe('EscapeSequenceParser', function (): void {
         parser.reset();
         testTerminal.clear();
         parser.currentState = ParserState.CSI_ENTRY;
-        parser.parse('\x7f');
+        parse(parser, '\x7f');
         chai.expect(parser.currentState).equal(ParserState.CSI_ENTRY);
         testTerminal.compare([]);
         parser.reset();
@@ -432,7 +447,7 @@ describe('EscapeSequenceParser', function (): void {
         const dispatches = r(0x40, 0x7f);
         for (let i = 0; i < dispatches.length; ++i) {
           parser.currentState = ParserState.CSI_ENTRY;
-          parser.parse(dispatches[i]);
+          parse(parser, dispatches[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([['csi', '', [0], dispatches[i]]]);
           parser.reset();
@@ -445,19 +460,19 @@ describe('EscapeSequenceParser', function (): void {
         const collect = ['\x3c', '\x3d', '\x3e', '\x3f'];
         for (let i = 0; i < params.length; ++i) {
           parser.currentState = ParserState.CSI_ENTRY;
-          parser.parse(params[i]);
+          parse(parser, params[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_PARAM);
           chai.expect(parser.params).eql([params[i].charCodeAt(0) - 48]);
           parser.reset();
         }
         parser.currentState = ParserState.CSI_ENTRY;
-        parser.parse('\x3b');
+        parse(parser, '\x3b');
         chai.expect(parser.currentState).equal(ParserState.CSI_PARAM);
         chai.expect(parser.params).eql([0, 0]);
         parser.reset();
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.CSI_ENTRY;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_PARAM);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -466,12 +481,12 @@ describe('EscapeSequenceParser', function (): void {
       it('state CSI_PARAM execute rules', function (): void {
         parser.reset();
         testTerminal.clear();
-        const exes = r(0x00, 0x18);
-        exes.concat(['\x19']);
-        exes.concat(r(0x1c, 0x20));
+        let exes = r(0x00, 0x18);
+        exes = exes.concat(['\x19']);
+        exes = exes.concat(r(0x1c, 0x20));
         for (let i = 0; i < exes.length; ++i) {
           parser.currentState = ParserState.CSI_PARAM;
-          parser.parse(exes[i]);
+          parse(parser, exes[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_PARAM);
           testTerminal.compare([['exe', exes[i]]]);
           parser.reset();
@@ -483,13 +498,13 @@ describe('EscapeSequenceParser', function (): void {
         const params = ['\x30', '\x31', '\x32', '\x33', '\x34', '\x35', '\x36', '\x37', '\x38', '\x39'];
         for (let i = 0; i < params.length; ++i) {
           parser.currentState = ParserState.CSI_PARAM;
-          parser.parse(params[i]);
+          parse(parser, params[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_PARAM);
           chai.expect(parser.params).eql([params[i].charCodeAt(0) - 48]);
           parser.reset();
         }
         parser.currentState = ParserState.CSI_PARAM;
-        parser.parse('\x3b');
+        parse(parser, '\x3b');
         chai.expect(parser.currentState).equal(ParserState.CSI_PARAM);
         chai.expect(parser.params).eql([0, 0]);
         parser.reset();
@@ -498,7 +513,7 @@ describe('EscapeSequenceParser', function (): void {
         parser.reset();
         testTerminal.clear();
         parser.currentState = ParserState.CSI_PARAM;
-        parser.parse('\x7f');
+        parse(parser, '\x7f');
         chai.expect(parser.currentState).equal(ParserState.CSI_PARAM);
         testTerminal.compare([]);
         parser.reset();
@@ -510,7 +525,7 @@ describe('EscapeSequenceParser', function (): void {
         for (let i = 0; i < dispatches.length; ++i) {
           parser.currentState = ParserState.CSI_PARAM;
           parser.params = [0, 1];
-          parser.parse(dispatches[i]);
+          parse(parser, dispatches[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([['csi', '', [0, 1], dispatches[i]]]);
           parser.reset();
@@ -522,7 +537,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.CSI_ENTRY;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -533,7 +548,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.CSI_PARAM;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -542,12 +557,12 @@ describe('EscapeSequenceParser', function (): void {
       it('state CSI_INTERMEDIATE execute rules', function (): void {
         parser.reset();
         testTerminal.clear();
-        const exes = r(0x00, 0x18);
-        exes.concat(['\x19']);
-        exes.concat(r(0x1c, 0x20));
+        let exes = r(0x00, 0x18);
+        exes = exes.concat(['\x19']);
+        exes = exes.concat(r(0x1c, 0x20));
         for (let i = 0; i < exes.length; ++i) {
           parser.currentState = ParserState.CSI_INTERMEDIATE;
-          parser.parse(exes[i]);
+          parse(parser, exes[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_INTERMEDIATE);
           testTerminal.compare([['exe', exes[i]]]);
           parser.reset();
@@ -559,7 +574,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.CSI_INTERMEDIATE;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -569,7 +584,7 @@ describe('EscapeSequenceParser', function (): void {
         parser.reset();
         testTerminal.clear();
         parser.currentState = ParserState.CSI_INTERMEDIATE;
-        parser.parse('\x7f');
+        parse(parser, '\x7f');
         chai.expect(parser.currentState).equal(ParserState.CSI_INTERMEDIATE);
         testTerminal.compare([]);
         parser.reset();
@@ -581,7 +596,7 @@ describe('EscapeSequenceParser', function (): void {
         for (let i = 0; i < dispatches.length; ++i) {
           parser.currentState = ParserState.CSI_INTERMEDIATE;
           parser.params = [0, 1];
-          parser.parse(dispatches[i]);
+          parse(parser, dispatches[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([['csi', '', [0, 1], dispatches[i]]]);
           parser.reset();
@@ -591,7 +606,7 @@ describe('EscapeSequenceParser', function (): void {
       it('trans CSI_ENTRY --> CSI_IGNORE', function (): void {
         parser.reset();
         parser.currentState = ParserState.CSI_ENTRY;
-        parser.parse('\x3a');
+        parse(parser, '\x3a');
         chai.expect(parser.currentState).equal(ParserState.CSI_IGNORE);
         parser.reset();
       });
@@ -600,7 +615,7 @@ describe('EscapeSequenceParser', function (): void {
         const chars = ['\x3a', '\x3c', '\x3d', '\x3e', '\x3f'];
         for (let i = 0; i < chars.length; ++i) {
           parser.currentState = ParserState.CSI_PARAM;
-          parser.parse('\x3b' + chars[i]);
+          parse(parser, '\x3b' + chars[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_IGNORE);
           chai.expect(parser.params).eql([0, 0]);
           parser.reset();
@@ -611,7 +626,7 @@ describe('EscapeSequenceParser', function (): void {
         const chars = r(0x30, 0x40);
         for (let i = 0; i < chars.length; ++i) {
           parser.currentState = ParserState.CSI_INTERMEDIATE;
-          parser.parse(chars[i]);
+          parse(parser, chars[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_IGNORE);
           chai.expect(parser.params).eql([0]);
           parser.reset();
@@ -620,12 +635,12 @@ describe('EscapeSequenceParser', function (): void {
       it('state CSI_IGNORE execute rules', function (): void {
         parser.reset();
         testTerminal.clear();
-        const exes = r(0x00, 0x18);
-        exes.concat(['\x19']);
-        exes.concat(r(0x1c, 0x20));
+        let exes = r(0x00, 0x18);
+        exes = exes.concat(['\x19']);
+        exes = exes.concat(r(0x1c, 0x20));
         for (let i = 0; i < exes.length; ++i) {
           parser.currentState = ParserState.CSI_IGNORE;
-          parser.parse(exes[i]);
+          parse(parser, exes[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_IGNORE);
           testTerminal.compare([['exe', exes[i]]]);
           parser.reset();
@@ -635,11 +650,11 @@ describe('EscapeSequenceParser', function (): void {
       it('state CSI_IGNORE ignore', function (): void {
         parser.reset();
         testTerminal.clear();
-        const ignored = r(0x20, 0x40);
-        ignored.concat(['\x7f']);
+        let ignored = r(0x20, 0x40);
+        ignored = ignored.concat(['\x7f']);
         for (let i = 0; i < ignored.length; ++i) {
           parser.currentState = ParserState.CSI_IGNORE;
-          parser.parse(ignored[i]);
+          parse(parser, ignored[i]);
           chai.expect(parser.currentState).equal(ParserState.CSI_IGNORE);
           testTerminal.compare([]);
           parser.reset();
@@ -652,7 +667,7 @@ describe('EscapeSequenceParser', function (): void {
         for (let i = 0; i < dispatches.length; ++i) {
           parser.currentState = ParserState.CSI_IGNORE;
           parser.params = [0, 1];
-          parser.parse(dispatches[i]);
+          parse(parser, dispatches[i]);
           chai.expect(parser.currentState).equal(ParserState.GROUND);
           testTerminal.compare([]);
           parser.reset();
@@ -664,7 +679,7 @@ describe('EscapeSequenceParser', function (): void {
         // C0
         let initializers = ['\x58', '\x5e', '\x5f'];
         for (let i = 0; i < initializers.length; ++i) {
-          parser.parse('\x1b' + initializers[i]);
+          parse(parser, '\x1b' + initializers[i]);
           chai.expect(parser.currentState).equal(ParserState.SOS_PM_APC_STRING);
           parser.reset();
         }
@@ -673,7 +688,7 @@ describe('EscapeSequenceParser', function (): void {
           parser.currentState = state;
           initializers = ['\x98', '\x9e', '\x9f'];
           for (let i = 0; i < initializers.length; ++i) {
-            parser.parse(initializers[i]);
+            parse(parser, initializers[i]);
             chai.expect(parser.currentState).equal(ParserState.SOS_PM_APC_STRING);
             parser.reset();
           }
@@ -681,13 +696,13 @@ describe('EscapeSequenceParser', function (): void {
       });
       it('state SOS_PM_APC_STRING ignore rules', function (): void {
         parser.reset();
-        const ignored = r(0x00, 0x18);
-        ignored.concat(['\x19']);
-        ignored.concat(r(0x1c, 0x20));
-        ignored.concat(r(0x20, 0x80));
+        let ignored = r(0x00, 0x18);
+        ignored = ignored.concat(['\x19']);
+        ignored = ignored.concat(r(0x1c, 0x20));
+        ignored = ignored.concat(r(0x20, 0x80));
         for (let i = 0; i < ignored.length; ++i) {
           parser.currentState = ParserState.SOS_PM_APC_STRING;
-          parser.parse(ignored[i]);
+          parse(parser, ignored[i]);
           chai.expect(parser.currentState).equal(ParserState.SOS_PM_APC_STRING);
           parser.reset();
         }
@@ -695,13 +710,13 @@ describe('EscapeSequenceParser', function (): void {
       it('trans ANYWHERE/ESCAPE --> OSC_STRING', function (): void {
         parser.reset();
         // C0
-        parser.parse('\x1b]');
+        parse(parser, '\x1b]');
         chai.expect(parser.currentState).equal(ParserState.OSC_STRING);
         parser.reset();
         // C1
         for (state in states) {
           parser.currentState = state;
-          parser.parse('\x9d');
+          parse(parser, '\x9d');
           chai.expect(parser.currentState).equal(ParserState.OSC_STRING);
           parser.reset();
         }
@@ -714,7 +729,7 @@ describe('EscapeSequenceParser', function (): void {
           '\x12', '\x13', '\x14', '\x15', '\x16', '\x17', '\x19', '\x1c', '\x1d', '\x1e', '\x1f'];
         for (let i = 0; i < ignored.length; ++i) {
           parser.currentState = ParserState.OSC_STRING;
-          parser.parse(ignored[i]);
+          parse(parser, ignored[i]);
           chai.expect(parser.currentState).equal(ParserState.OSC_STRING);
           chai.expect(parser.osc).equal('');
           parser.reset();
@@ -725,7 +740,7 @@ describe('EscapeSequenceParser', function (): void {
         const puts = r(0x20, 0x80);
         for (let i = 0; i < puts.length; ++i) {
           parser.currentState = ParserState.OSC_STRING;
-          parser.parse(puts[i]);
+          parse(parser, puts[i]);
           chai.expect(parser.currentState).equal(ParserState.OSC_STRING);
           chai.expect(parser.osc).equal(puts[i]);
           parser.reset();
@@ -734,13 +749,13 @@ describe('EscapeSequenceParser', function (): void {
       it('state DCS_ENTRY', function (): void {
         parser.reset();
         // C0
-        parser.parse('\x1bP');
+        parse(parser, '\x1bP');
         chai.expect(parser.currentState).equal(ParserState.DCS_ENTRY);
         parser.reset();
         // C1
         for (state in states) {
           parser.currentState = state;
-          parser.parse('\x90');
+          parse(parser, '\x90');
           chai.expect(parser.currentState).equal(ParserState.DCS_ENTRY);
           parser.reset();
         }
@@ -753,7 +768,7 @@ describe('EscapeSequenceParser', function (): void {
           '\x12', '\x13', '\x14', '\x15', '\x16', '\x17', '\x19', '\x1c', '\x1d', '\x1e', '\x1f', '\x7f'];
         for (let i = 0; i < ignored.length; ++i) {
           parser.currentState = ParserState.DCS_ENTRY;
-          parser.parse(ignored[i]);
+          parse(parser, ignored[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_ENTRY);
           parser.reset();
         }
@@ -764,19 +779,19 @@ describe('EscapeSequenceParser', function (): void {
         const collect = ['\x3c', '\x3d', '\x3e', '\x3f'];
         for (let i = 0; i < params.length; ++i) {
           parser.currentState = ParserState.DCS_ENTRY;
-          parser.parse(params[i]);
+          parse(parser, params[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PARAM);
           chai.expect(parser.params).eql([params[i].charCodeAt(0) - 48]);
           parser.reset();
         }
         parser.currentState = ParserState.DCS_ENTRY;
-        parser.parse('\x3b');
+        parse(parser, '\x3b');
         chai.expect(parser.currentState).equal(ParserState.DCS_PARAM);
         chai.expect(parser.params).eql([0, 0]);
         parser.reset();
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.DCS_ENTRY;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PARAM);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -790,7 +805,7 @@ describe('EscapeSequenceParser', function (): void {
           '\x12', '\x13', '\x14', '\x15', '\x16', '\x17', '\x19', '\x1c', '\x1d', '\x1e', '\x1f', '\x7f'];
         for (let i = 0; i < ignored.length; ++i) {
           parser.currentState = ParserState.DCS_PARAM;
-          parser.parse(ignored[i]);
+          parse(parser, ignored[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PARAM);
           parser.reset();
         }
@@ -800,13 +815,13 @@ describe('EscapeSequenceParser', function (): void {
         const params = ['\x30', '\x31', '\x32', '\x33', '\x34', '\x35', '\x36', '\x37', '\x38', '\x39'];
         for (let i = 0; i < params.length; ++i) {
           parser.currentState = ParserState.DCS_PARAM;
-          parser.parse(params[i]);
+          parse(parser, params[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PARAM);
           chai.expect(parser.params).eql([params[i].charCodeAt(0) - 48]);
           parser.reset();
         }
         parser.currentState = ParserState.DCS_PARAM;
-        parser.parse('\x3b');
+        parse(parser, '\x3b');
         chai.expect(parser.currentState).equal(ParserState.DCS_PARAM);
         chai.expect(parser.params).eql([0, 0]);
         parser.reset();
@@ -814,7 +829,7 @@ describe('EscapeSequenceParser', function (): void {
       it('trans DCS_ENTRY --> DCS_IGNORE', function (): void {
         parser.reset();
         parser.currentState = ParserState.DCS_ENTRY;
-        parser.parse('\x3a');
+        parse(parser, '\x3a');
         chai.expect(parser.currentState).equal(ParserState.DCS_IGNORE);
         parser.reset();
       });
@@ -823,7 +838,7 @@ describe('EscapeSequenceParser', function (): void {
         const chars = ['\x3a', '\x3c', '\x3d', '\x3e', '\x3f'];
         for (let i = 0; i < chars.length; ++i) {
           parser.currentState = ParserState.DCS_PARAM;
-          parser.parse('\x3b' + chars[i]);
+          parse(parser, '\x3b' + chars[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_IGNORE);
           chai.expect(parser.params).eql([0, 0]);
           parser.reset();
@@ -834,21 +849,21 @@ describe('EscapeSequenceParser', function (): void {
         const chars = r(0x30, 0x40);
         for (let i = 0; i < chars.length; ++i) {
           parser.currentState = ParserState.DCS_INTERMEDIATE;
-          parser.parse(chars[i]);
+          parse(parser, chars[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_IGNORE);
           parser.reset();
         }
       });
       it('state DCS_IGNORE ignore rules', function (): void {
         parser.reset();
-        const ignored = [
+        let ignored = [
           '\x00', '\x01', '\x02', '\x03', '\x04', '\x05', '\x06', '\x07', '\x08',
           '\x09', '\x0a', '\x0b', '\x0c', '\x0d', '\x0e', '\x0f', '\x10', '\x11',
           '\x12', '\x13', '\x14', '\x15', '\x16', '\x17', '\x19', '\x1c', '\x1d', '\x1e', '\x1f', '\x7f'];
-        ignored.concat(r(0x20, 0x80));
+        ignored = ignored.concat(r(0x20, 0x80));
         for (let i = 0; i < ignored.length; ++i) {
           parser.currentState = ParserState.DCS_IGNORE;
-          parser.parse(ignored[i]);
+          parse(parser, ignored[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_IGNORE);
           parser.reset();
         }
@@ -858,7 +873,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.DCS_ENTRY;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -869,7 +884,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.DCS_PARAM;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -883,7 +898,7 @@ describe('EscapeSequenceParser', function (): void {
           '\x12', '\x13', '\x14', '\x15', '\x16', '\x17', '\x19', '\x1c', '\x1d', '\x1e', '\x1f', '\x7f'];
         for (let i = 0; i < ignored.length; ++i) {
           parser.currentState = ParserState.DCS_INTERMEDIATE;
-          parser.parse(ignored[i]);
+          parse(parser, ignored[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_INTERMEDIATE);
           parser.reset();
         }
@@ -893,7 +908,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x20, 0x30);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.DCS_INTERMEDIATE;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_INTERMEDIATE);
           chai.expect(parser.collect).equal(collect[i]);
           parser.reset();
@@ -904,7 +919,7 @@ describe('EscapeSequenceParser', function (): void {
         const chars = r(0x30, 0x40);
         for (let i = 0; i < chars.length; ++i) {
           parser.currentState = ParserState.DCS_INTERMEDIATE;
-          parser.parse('\x20' + chars[i]);
+          parse(parser, '\x20' + chars[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_IGNORE);
           chai.expect(parser.collect).equal('\x20');
           parser.reset();
@@ -916,7 +931,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x40, 0x7f);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.DCS_ENTRY;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PASSTHROUGH);
           testTerminal.compare([['dcs hook', '', [0], collect[i]]]);
           parser.reset();
@@ -929,7 +944,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x40, 0x7f);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.DCS_PARAM;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PASSTHROUGH);
           testTerminal.compare([['dcs hook', '', [0], collect[i]]]);
           parser.reset();
@@ -942,7 +957,7 @@ describe('EscapeSequenceParser', function (): void {
         const collect = r(0x40, 0x7f);
         for (let i = 0; i < collect.length; ++i) {
           parser.currentState = ParserState.DCS_INTERMEDIATE;
-          parser.parse(collect[i]);
+          parse(parser, collect[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PASSTHROUGH);
           testTerminal.compare([['dcs hook', '', [0], collect[i]]]);
           parser.reset();
@@ -952,14 +967,14 @@ describe('EscapeSequenceParser', function (): void {
       it('state DCS_PASSTHROUGH put action', function (): void {
         parser.reset();
         testTerminal.clear();
-        const puts = r(0x00, 0x18);
-        puts.concat(['\x19']);
-        puts.concat(r(0x1c, 0x20));
-        puts.concat(r(0x20, 0x7f));
+        let puts = r(0x00, 0x18);
+        puts = puts.concat(['\x19']);
+        puts = puts.concat(r(0x1c, 0x20));
+        puts = puts.concat(r(0x20, 0x7f));
         for (let i = 0; i < puts.length; ++i) {
           parser.currentState = ParserState.DCS_PASSTHROUGH;
           parser.mockActiveDcsHandler();
-          parser.parse(puts[i]);
+          parse(parser, puts[i]);
           chai.expect(parser.currentState).equal(ParserState.DCS_PASSTHROUGH);
           testTerminal.compare([['dcs put', puts[i]]]);
           parser.reset();
@@ -970,7 +985,7 @@ describe('EscapeSequenceParser', function (): void {
         parser.reset();
         testTerminal.clear();
         parser.currentState = ParserState.DCS_PASSTHROUGH;
-        parser.parse('\x7f');
+        parse(parser, '\x7f');
         chai.expect(parser.currentState).equal(ParserState.DCS_PASSTHROUGH);
         testTerminal.compare([]);
         parser.reset();
@@ -989,7 +1004,7 @@ describe('EscapeSequenceParser', function (): void {
             parser.reset();
             testTerminal.clear();
           }
-          parser.parse(s);
+          parse(parser, s);
           testTerminal.compare(value);
         };
       });
@@ -1067,7 +1082,7 @@ describe('EscapeSequenceParser', function (): void {
       parser.reset();
       testTerminal.clear();
       parser.currentState = ParserState.CSI_IGNORE;
-      parser.parse('€öäü');
+      parse(parser, '€öäü');
       chai.expect(parser.currentState).equal(ParserState.CSI_IGNORE);
       testTerminal.compare([]);
       parser.reset();
@@ -1077,7 +1092,7 @@ describe('EscapeSequenceParser', function (): void {
       parser.reset();
       testTerminal.clear();
       parser.currentState = ParserState.DCS_IGNORE;
-      parser.parse('€öäü');
+      parse(parser, '€öäü');
       chai.expect(parser.currentState).equal(ParserState.DCS_IGNORE);
       testTerminal.compare([]);
       parser.reset();
@@ -1087,7 +1102,7 @@ describe('EscapeSequenceParser', function (): void {
       parser.reset();
       testTerminal.clear();
       parser.currentState = ParserState.DCS_PASSTHROUGH;
-      parser.parse('\x901;2;3+$a€öäü');
+      parse(parser, '\x901;2;3+$a€öäü');
       chai.expect(parser.currentState).equal(ParserState.DCS_PASSTHROUGH);
       testTerminal.compare([['dcs hook', '+$', [1, 2, 3], 'a'], ['dcs put', '€öäü']]);
       parser.reset();
@@ -1097,7 +1112,7 @@ describe('EscapeSequenceParser', function (): void {
       parser.reset();
       testTerminal.clear();
       parser.currentState = ParserState.GROUND;
-      parser.parse('\x9c');
+      parse(parser, '\x9c');
       chai.expect(parser.currentState).equal(ParserState.GROUND);
       testTerminal.compare([]);
       parser.reset();
@@ -1127,15 +1142,17 @@ describe('EscapeSequenceParser', function (): void {
       clearAccu();
     });
     it('print handler', function (): void {
-      parser2.setPrintHandler(function (data: string, start: number, end: number): void {
-        print += data.substring(start, end);
+      parser2.setPrintHandler(function (data: Uint32Array, start: number, end: number): void {
+        for (let i = start; i < end; ++i) {
+          print += stringFromCodePoint(data[i]);
+        }
       });
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(print).equal('hello world!$>');
       parser2.clearPrintHandler();
       parser2.clearPrintHandler(); // should not throw
       clearAccu();
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(print).equal('');
     });
     it('ESC handler', function (): void {
@@ -1145,28 +1162,28 @@ describe('EscapeSequenceParser', function (): void {
       parser2.setEscHandler('E', function (): void {
         esc.push('E');
       });
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(esc).eql(['%G', 'E']);
       parser2.clearEscHandler('%G');
       parser2.clearEscHandler('%G'); // should not throw
       clearAccu();
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(esc).eql(['E']);
       parser2.clearEscHandler('E');
       clearAccu();
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(esc).eql([]);
     });
     it('CSI handler', function (): void {
       parser2.setCsiHandler('m', function (params: number[], collect: string): void {
         csi.push(['m', params, collect]);
       });
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(csi).eql([['m', [1, 31], ''], ['m', [0], '']]);
       parser2.clearCsiHandler('m');
       parser2.clearCsiHandler('m'); // should not throw
       clearAccu();
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(csi).eql([]);
     });
     describe('CSI custom handlers', () => {
@@ -1174,7 +1191,7 @@ describe('EscapeSequenceParser', function (): void {
         const csiCustom: [string, number[], string][] = [];
         parser2.setCsiHandler('m', (params, collect) => csi.push(['m', params, collect]));
         parser2.addCsiHandler('m', (params, collect) => { csiCustom.push(['m', params, collect]); return true; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(csi).eql([], 'Should not fallback to original handler');
         chai.expect(csiCustom).eql([['m', [1, 31], ''], ['m', [0], '']]);
       });
@@ -1182,7 +1199,7 @@ describe('EscapeSequenceParser', function (): void {
         const csiCustom: [string, number[], string][] = [];
         parser2.setCsiHandler('m', (params, collect) => csi.push(['m', params, collect]));
         parser2.addCsiHandler('m', (params, collect) => { csiCustom.push(['m', params, collect]); return false; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(csi).eql([['m', [1, 31], ''], ['m', [0], '']], 'Should fallback to original handler');
         chai.expect(csiCustom).eql([['m', [1, 31], ''], ['m', [0], '']]);
       });
@@ -1192,7 +1209,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setCsiHandler('m', (params, collect) => csi.push(['m', params, collect]));
         parser2.addCsiHandler('m', (params, collect) => { csiCustom.push(['m', params, collect]); return true; });
         parser2.addCsiHandler('m', (params, collect) => { csiCustom2.push(['m', params, collect]); return false; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(csi).eql([], 'Should not fallback to original handler');
         chai.expect(csiCustom).eql([['m', [1, 31], ''], ['m', [0], '']]);
         chai.expect(csiCustom2).eql([['m', [1, 31], ''], ['m', [0], '']]);
@@ -1203,7 +1220,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setCsiHandler('m', (params, collect) => csi.push(['m', params, collect]));
         parser2.addCsiHandler('m', (params, collect) => { csiCustom.push(['m', params, collect]); return true; });
         parser2.addCsiHandler('m', (params, collect) => { csiCustom2.push(['m', params, collect]); return true; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(csi).eql([], 'Should not fallback to original handler');
         chai.expect(csiCustom).eql([], 'Should not fallback once');
         chai.expect(csiCustom2).eql([['m', [1, 31], ''], ['m', [0], '']]);
@@ -1213,7 +1230,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setCsiHandler('m', () => order.push(1));
         parser2.addCsiHandler('m', () => { order.push(2); return false; });
         parser2.addCsiHandler('m', () => { order.push(3); return false; });
-        parser2.parse('\x1b[0m');
+        parse(parser2, '\x1b[0m');
         chai.expect(order).eql([3, 2, 1]);
       });
       it('Dispose should work', () => {
@@ -1221,7 +1238,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setCsiHandler('m', (params, collect) => csi.push(['m', params, collect]));
         const customHandler = parser2.addCsiHandler('m', (params, collect) => { csiCustom.push(['m', params, collect]); return true; });
         customHandler.dispose();
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(csi).eql([['m', [1, 31], ''], ['m', [0], '']]);
         chai.expect(csiCustom).eql([], 'Should not use custom handler as it was disposed');
       });
@@ -1231,7 +1248,7 @@ describe('EscapeSequenceParser', function (): void {
         const customHandler = parser2.addCsiHandler('m', (params, collect) => { csiCustom.push(['m', params, collect]); return true; });
         customHandler.dispose();
         customHandler.dispose();
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(csi).eql([['m', [1, 31], ''], ['m', [0], '']]);
         chai.expect(csiCustom).eql([], 'Should not use custom handler as it was disposed');
       });
@@ -1243,24 +1260,24 @@ describe('EscapeSequenceParser', function (): void {
       parser2.setExecuteHandler('\r', function (): void {
         exe.push('\r');
       });
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(exe).eql(['\r', '\n']);
       parser2.clearExecuteHandler('\r');
       parser2.clearExecuteHandler('\r'); // should not throw
       clearAccu();
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(exe).eql(['\n']);
     });
     it('OSC handler', function (): void {
       parser2.setOscHandler(1, function (data: string): void {
         osc.push([1, data]);
       });
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(osc).eql([[1, 'foo=bar']]);
       parser2.clearOscHandler(1);
       parser2.clearOscHandler(1); // should not throw
       clearAccu();
-      parser2.parse(INPUT);
+      parse(parser2, INPUT);
       chai.expect(osc).eql([]);
     });
     describe('OSC custom handlers', () => {
@@ -1268,7 +1285,7 @@ describe('EscapeSequenceParser', function (): void {
         const oscCustom: [number, string][] = [];
         parser2.setOscHandler(1, data => osc.push([1, data]));
         parser2.addOscHandler(1, data => { oscCustom.push([1, data]); return true; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(osc).eql([], 'Should not fallback to original handler');
         chai.expect(oscCustom).eql([[1, 'foo=bar']]);
       });
@@ -1276,7 +1293,7 @@ describe('EscapeSequenceParser', function (): void {
         const oscCustom: [number, string][] = [];
         parser2.setOscHandler(1, data => osc.push([1, data]));
         parser2.addOscHandler(1, data => { oscCustom.push([1, data]); return false; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(osc).eql([[1, 'foo=bar']], 'Should fallback to original handler');
         chai.expect(oscCustom).eql([[1, 'foo=bar']]);
       });
@@ -1286,7 +1303,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setOscHandler(1, data => osc.push([1, data]));
         parser2.addOscHandler(1, data => { oscCustom.push([1, data]); return true; });
         parser2.addOscHandler(1, data => { oscCustom2.push([1, data]); return false; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(osc).eql([], 'Should not fallback to original handler');
         chai.expect(oscCustom).eql([[1, 'foo=bar']]);
         chai.expect(oscCustom2).eql([[1, 'foo=bar']]);
@@ -1297,7 +1314,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setOscHandler(1, data => osc.push([1, data]));
         parser2.addOscHandler(1, data => { oscCustom.push([1, data]); return true; });
         parser2.addOscHandler(1, data => { oscCustom2.push([1, data]); return true; });
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(osc).eql([], 'Should not fallback to original handler');
         chai.expect(oscCustom).eql([], 'Should not fallback once');
         chai.expect(oscCustom2).eql([[1, 'foo=bar']]);
@@ -1307,7 +1324,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setOscHandler(1, () => order.push(1));
         parser2.addOscHandler(1, () => { order.push(2); return false; });
         parser2.addOscHandler(1, () => { order.push(3); return false; });
-        parser2.parse('\x1b]1;foo=bar\x1b\\');
+        parse(parser2, '\x1b]1;foo=bar\x1b\\');
         chai.expect(order).eql([3, 2, 1]);
       });
       it('Dispose should work', () => {
@@ -1315,7 +1332,7 @@ describe('EscapeSequenceParser', function (): void {
         parser2.setOscHandler(1, data => osc.push([1, data]));
         const customHandler = parser2.addOscHandler(1, data => { oscCustom.push([1, data]); return true; });
         customHandler.dispose();
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(osc).eql([[1, 'foo=bar']]);
         chai.expect(oscCustom).eql([], 'Should not use custom handler as it was disposed');
       });
@@ -1325,7 +1342,7 @@ describe('EscapeSequenceParser', function (): void {
         const customHandler = parser2.addOscHandler(1, data => { oscCustom.push([1, data]); return true; });
         customHandler.dispose();
         customHandler.dispose();
-        parser2.parse(INPUT);
+        parse(parser2, INPUT);
         chai.expect(osc).eql([[1, 'foo=bar']]);
         chai.expect(oscCustom).eql([], 'Should not use custom handler as it was disposed');
       });
@@ -1335,15 +1352,19 @@ describe('EscapeSequenceParser', function (): void {
         hook: function (collect: string, params: number[], flag: number): void {
           dcs.push(['hook', collect, params, flag]);
         },
-        put: function (data: string, start: number, end: number): void {
-          dcs.push(['put', data.substring(start, end)]);
+        put: function (data: Uint32Array, start: number, end: number): void {
+          let s = '';
+          for (let i = start; i < end; ++i) {
+            s += stringFromCodePoint(data[i]);
+          }
+          dcs.push(['put', s]);
         },
         unhook: function (): void {
           dcs.push(['unhook']);
         }
       });
-      parser2.parse('\x1bP1;2;3+pabc');
-      parser2.parse(';de\x9c');
+      parse(parser2, '\x1bP1;2;3+pabc');
+      parse(parser2, ';de\x9c');
       chai.expect(dcs).eql([
         ['hook', '+', [1, 2, 3], 'p'.charCodeAt(0)],
         ['put', 'abc'], ['put', ';de'],
@@ -1352,8 +1373,8 @@ describe('EscapeSequenceParser', function (): void {
       parser2.clearDcsHandler('+p');
       parser2.clearDcsHandler('+p'); // should not throw
       clearAccu();
-      parser2.parse('\x1bP1;2;3+pabc');
-      parser2.parse(';de\x9c');
+      parse(parser2, '\x1bP1;2;3+pabc');
+      parse(parser2, ';de\x9c');
       chai.expect(dcs).eql([]);
     });
     it('ERROR handler', function (): void {
@@ -1362,7 +1383,7 @@ describe('EscapeSequenceParser', function (): void {
         errorState = state;
         return state;
       });
-      parser2.parse('\x1b[1;2;€;3m'); // faulty escape sequence
+      parse(parser2, '\x1b[1;2;€;3m'); // faulty escape sequence
       chai.expect(errorState).eql({
         position: 6,
         code: '€'.charCodeAt(0),
@@ -1377,7 +1398,7 @@ describe('EscapeSequenceParser', function (): void {
       parser2.clearErrorHandler();
       parser2.clearErrorHandler(); // should not throw
       errorState = null;
-      parser2.parse('\x1b[1;2;a;3m');
+      parse(parser2, '\x1b[1;2;a;3m');
       chai.expect(errorState).eql(null);
     });
   });

src/EscapeSequenceParser.ts

@@ -6,6 +6,7 @@
 import { ParserState, ParserAction, IParsingState, IDcsHandler, IEscapeSequenceParser } from './Types';
 import { IDisposable } from 'xterm';
 import { Disposable } from './common/Lifecycle';
+import { utf32ToString } from './core/input/TextDecoder';
 
 interface IHandlerCollection<T> {
   [key: string]: T[];
@@ -134,6 +135,7 @@ export const VT500_TRANSITION_TABLE = (function (): TransitionTable {
   table.addMany(PRINTABLES, ParserState.SOS_PM_APC_STRING, ParserAction.IGNORE, ParserState.SOS_PM_APC_STRING);
   table.addMany(EXECUTABLES, ParserState.SOS_PM_APC_STRING, ParserAction.IGNORE, ParserState.SOS_PM_APC_STRING);
   table.add(0x9c, ParserState.SOS_PM_APC_STRING, ParserAction.IGNORE, ParserState.GROUND);
+  table.add(0x7f, ParserState.SOS_PM_APC_STRING, ParserAction.IGNORE, ParserState.SOS_PM_APC_STRING);
   // csi entries
   table.add(0x5b, ParserState.ESCAPE, ParserAction.CLEAR, ParserState.CSI_ENTRY);
   table.addMany(r(0x40, 0x7f), ParserState.CSI_ENTRY, ParserAction.CSI_DISPATCH, ParserState.GROUND);
@@ -202,7 +204,7 @@ export const VT500_TRANSITION_TABLE = (function (): TransitionTable {
  */
 class DcsDummy implements IDcsHandler {
   hook(collect: string, params: number[], flag: number): void { }
-  put(data: string, start: number, end: number): void { }
+  put(data: Uint32Array, start: number, end: number): void { }
   unhook(): void { }
 }
 
@@ -228,7 +230,7 @@ export class EscapeSequenceParser extends Disposable implements IEscapeSequenceP
   protected _collect: string;
 
   // handler lookup containers
-  protected _printHandler: (data: string, start: number, end: number) => void;
+  protected _printHandler: (data: Uint32Array, start: number, end: number) => void;
   protected _executeHandlers: any;
   protected _csiHandlers: IHandlerCollection<CsiHandler>;
   protected _escHandlers: any;
@@ -238,7 +240,7 @@ export class EscapeSequenceParser extends Disposable implements IEscapeSequenceP
   protected _errorHandler: (state: IParsingState) => IParsingState;
 
   // fallback handlers
-  protected _printHandlerFb: (data: string, start: number, end: number) => void;
+  protected _printHandlerFb: (data: Uint32Array, start: number, end: number) => void;
   protected _executeHandlerFb: (code: number) => void;
   protected _csiHandlerFb: (collect: string, params: number[], flag: number) => void;
   protected _escHandlerFb: (collect: string, flag: number) => void;
@@ -294,7 +296,7 @@ export class EscapeSequenceParser extends Disposable implements IEscapeSequenceP
     this._errorHandler = null;
   }
 
-  setPrintHandler(callback: (data: string, start: number, end: number) => void): void {
+  setPrintHandler(callback: (data: Uint32Array, start: number, end: number) => void): void {
     this._printHandler = callback;
   }
   clearPrintHandler(): void {
@@ -397,7 +399,7 @@ export class EscapeSequenceParser extends Disposable implements IEscapeSequenceP
     this._activeDcsHandler = null;
   }
 
-  parse(data: string): void {
+  parse(data: Uint32Array, length: number): void {
     let code = 0;
     let transition = 0;
     let error = false;
@@ -412,15 +414,14 @@ export class EscapeSequenceParser extends Disposable implements IEscapeSequenceP
     let callback: Function | null = null;
 
     // process input string
-    const l = data.length;
-    for (let i = 0; i < l; ++i) {
-      code = data.charCodeAt(i);
+    for (let i = 0; i < length; ++i) {
+      code = data[i];
 
       // shortcut for most chars (print action)
       if (currentState === ParserState.GROUND && code > 0x1f && code < 0x80) {
         print = (~print) ? print : i;
         do i++;
-        while (i < l && data.charCodeAt(i) > 0x1f && data.charCodeAt(i) < 0x80);
+        while (i < length && data[i] > 0x1f && data[i] < 0x80);
         i--;
         continue;
       }
@@ -563,10 +564,10 @@ export class EscapeSequenceParser extends Disposable implements IEscapeSequenceP
           break;
         case ParserAction.OSC_PUT:
           for (let j = i + 1; ; j++) {
-            if (j >= l
-                || (code = data.charCodeAt(j)) < 0x20
+            if (j >= length
+                || (code = data[j]) < 0x20
                 || (code > 0x7f && code <= 0x9f)) {
-              osc += data.substring(i, j);
+              osc += utf32ToString(data, i, j);
               i = j - 1;
               break;
             }
@@ -610,9 +611,9 @@ export class EscapeSequenceParser extends Disposable implements IEscapeSequenceP
 
     // push leftover pushable buffers to terminal
     if (currentState === ParserState.GROUND && ~print) {
-      this._printHandler(data, print, data.length);
+      this._printHandler(data, print, length);
     } else if (currentState === ParserState.DCS_PASSTHROUGH && ~dcs && dcsHandler) {
-      dcsHandler.put(data, dcs, data.length);
+      dcsHandler.put(data, dcs, length);
     }
 
     // save non pushable buffers

src/InputHandler.test.ts

@@ -337,7 +337,9 @@ describe('InputHandler', () => {
     it('should not cause an infinite loop (regression test)', () => {
       const term = new Terminal();
       const inputHandler = new InputHandler(term);
-      inputHandler.print(String.fromCharCode(0x200B), 0, 1);
+      const container = new Uint32Array(10);
+      container[0] = 0x200B;
+      inputHandler.print(container, 0, 1);
     });
   });
 

src/InputHandler.ts

@@ -14,6 +14,8 @@ import { EscapeSequenceParser } from './EscapeSequenceParser';
 import { ICharset } from './core/Types';
 import { IDisposable } from 'xterm';
 import { Disposable } from './common/Lifecycle';
+import { concat } from './common/TypedArrayUtils';
+import { StringToUtf32, stringFromCodePoint, utf32ToString } from './core/input/TextDecoder';
 
 /**
  * Map collect to glevel. Used in `selectCharset`.
@@ -25,26 +27,6 @@ const GLEVEL: {[key: string]: number} = {'(': 0, ')': 1, '*': 2, '+': 3, '-': 1,
  * DCS subparser implementations
  */
 
- /**
-  * DCS + q Pt ST (xterm)
-  *   Request Terminfo String
-  *   not supported
-  */
-class RequestTerminfo implements IDcsHandler {
-  private _data: string;
-  constructor(private _terminal: any) { }
-  hook(collect: string, params: number[], flag: number): void {
-    this._data = '';
-  }
-  put(data: string, start: number, end: number): void {
-    this._data += data.substring(start, end);
-  }
-  unhook(): void {
-    // invalid: DCS 0 + r Pt ST
-    this._terminal.handler(`${C0.ESC}P0+r${this._data}${C0.ESC}\\`);
-  }
-}
-
 /**
  * DCS $ q Pt ST
  *   DECRQSS (https://vt100.net/docs/vt510-rm/DECRQSS.html)
@@ -52,21 +34,22 @@ class RequestTerminfo implements IDcsHandler {
  *   Response: DECRPSS (https://vt100.net/docs/vt510-rm/DECRPSS.html)
  */
 class DECRQSS implements IDcsHandler {
-  private _data: string;
+  private _data: Uint32Array = new Uint32Array(0);
 
   constructor(private _terminal: any) { }
 
   hook(collect: string, params: number[], flag: number): void {
-    // reset data
-    this._data = '';
+    this._data = new Uint32Array(0);
   }
 
-  put(data: string, start: number, end: number): void {
-    this._data += data.substring(start, end);
+  put(data: Uint32Array, start: number, end: number): void {
+    this._data = concat(this._data, data.subarray(start, end));
   }
 
   unhook(): void {
-    switch (this._data) {
+    const data = utf32ToString(this._data);
+    this._data = new Uint32Array(0);
+    switch (data) {
       // valid: DCS 1 $ r Pt ST (xterm)
       case '"q': // DECSCA
         return this._terminal.handler(`${C0.ESC}P1$r0"q${C0.ESC}\\`);
@@ -86,8 +69,8 @@ class DECRQSS implements IDcsHandler {
         return this._terminal.handler(`${C0.ESC}P1$r${style} q${C0.ESC}\\`);
       default:
         // invalid: DCS 0 $ r Pt ST (xterm)
-        this._terminal.error('Unknown DCS $q %s', this._data);
-        this._terminal.handler(`${C0.ESC}P0$r${this._data}${C0.ESC}\\`);
+        this._terminal.error('Unknown DCS $q %s', data);
+        this._terminal.handler(`${C0.ESC}P0$r${C0.ESC}\\`);
     }
   }
 }
@@ -98,11 +81,17 @@ class DECRQSS implements IDcsHandler {
  *   not supported
  */
 
- /**
-  * DCS + p Pt ST (xterm)
-  *   Set Terminfo Data
-  *   not supported
-  */
+/**
+ * DCS + q Pt ST (xterm)
+ *   Request Terminfo String
+ *   not implemented
+ */
+
+/**
+ * DCS + p Pt ST (xterm)
+ *   Set Terminfo Data
+ *   not supported
+ */
 
 
 
@@ -114,7 +103,8 @@ class DECRQSS implements IDcsHandler {
  * each function's header comment.
  */
 export class InputHandler extends Disposable implements IInputHandler {
-  private _surrogateFirst: string;
+  private _parseBuffer: Uint32Array = new Uint32Array(4096);
+  private _stringDecoder: StringToUtf32 = new StringToUtf32();
 
   constructor(
       protected _terminal: IInputHandlingTerminal,
@@ -124,8 +114,6 @@ export class InputHandler extends Disposable implements IInputHandler {
 
     this.register(this._parser);
 
-    this._surrogateFirst = '';
-
     /**
      * custom fallback handlers
      */
@@ -288,7 +276,6 @@ export class InputHandler extends Disposable implements IInputHandler {
      * DCS handler
      */
     this._parser.setDcsHandler('$q', new DECRQSS(this._terminal));
-    this._parser.setDcsHandler('+q', new RequestTerminfo(this._terminal));
   }
 
   public dispose(): void {
@@ -311,13 +298,13 @@ export class InputHandler extends Disposable implements IInputHandler {
       this._terminal.log('data: ' + data);
     }
 
-    // apply leftover surrogate high from last write
-    if (this._surrogateFirst) {
-      data = this._surrogateFirst + data;
-      this._surrogateFirst = '';
+    if (this._parseBuffer.length < data.length) {
+      this._parseBuffer = new Uint32Array(data.length);
     }
-
-    this._parser.parse(data);
+    for (let i = 0; i < data.length; ++i) {
+      this._parseBuffer[i] = data.charCodeAt(i);
+    }
+    this._parser.parse(this._parseBuffer, this._stringDecoder.decode(data, this._parseBuffer));
 
     buffer = this._terminal.buffer;
     if (buffer.x !== cursorStartX || buffer.y !== cursorStartY) {
@@ -325,9 +312,9 @@ export class InputHandler extends Disposable implements IInputHandler {
     }
   }
 
-  public print(data: string, start: number, end: number): void {
-    let char: string;
+  public print(data: Uint32Array, start: number, end: number): void {
     let code: number;
+    let char: string;
     let chWidth: number;
     const buffer: IBuffer = this._terminal.buffer;
     const charset: ICharset = this._terminal.charset;
@@ -339,41 +326,23 @@ export class InputHandler extends Disposable implements IInputHandler {
     let bufferRow = buffer.lines.get(buffer.y + buffer.ybase);
 
     this._terminal.updateRange(buffer.y);
-    for (let stringPosition = start; stringPosition < end; ++stringPosition) {
-      char = data.charAt(stringPosition);
-      code = data.charCodeAt(stringPosition);
-
-      // surrogate pair handling
-      if (0xD800 <= code && code <= 0xDBFF) {
-        if (++stringPosition >= end) {
-          // end of input:
-          // handle pairs as true UTF-16 and wait for the second part
-          // since we expect the input comming from a stream there is
-          // a small chance that the surrogate pair got split
-          // therefore we dont process the first char here, instead
-          // it gets added as first char to the next processed chunk
-          this._surrogateFirst = char;
-          continue;
-        }
-        const second = data.charCodeAt(stringPosition);
-        // if the second part is in surrogate pair range create the high codepoint
-        // otherwise fall back to UCS-2 behavior (handle codepoints independently)
-        if (0xDC00 <= second && second <= 0xDFFF) {
-          code = (code - 0xD800) * 0x400 + second - 0xDC00 + 0x10000;
-          char += data.charAt(stringPosition);
-        } else {
-          stringPosition--;
-        }
-      }
+    for (let pos = start; pos < end; ++pos) {
+      code = data[pos];
+      char = stringFromCodePoint(code);
 
       // calculate print space
       // expensive call, therefore we save width in line buffer
       chWidth = wcwidth(code);
 
       // get charset replacement character
-      if (charset) {
-        char = charset[char] || char;
-        code = char.charCodeAt(0);
+      // charset are only defined for ASCII, therefore we only
+      // search for an replacement char if code < 127
+      if (code < 127 && charset) {
+        const ch = charset[char];
+        if (ch) {
+          code = ch.charCodeAt(0);
+          char = ch;
+        }
       }
 
       if (screenReaderMode) {
@@ -1312,8 +1281,12 @@ export class InputHandler extends Disposable implements IInputHandler {
           this._terminal.vt200Mouse = params[0] === 1000;
           this._terminal.normalMouse = params[0] > 1000;
           this._terminal.mouseEvents = true;
-          this._terminal.element.classList.add('enable-mouse-events');
-          this._terminal.selectionManager.disable();
+          if (this._terminal.element) {
+            this._terminal.element.classList.add('enable-mouse-events');
+          }
+          if (this._terminal.selectionManager) {
+            this._terminal.selectionManager.disable();
+          }
           this._terminal.log('Binding to mouse events.');
           break;
         case 1004: // send focusin/focusout events
@@ -1500,8 +1473,12 @@ export class InputHandler extends Disposable implements IInputHandler {
           this._terminal.vt200Mouse = false;
           this._terminal.normalMouse = false;
           this._terminal.mouseEvents = false;
-          this._terminal.element.classList.remove('enable-mouse-events');
-          this._terminal.selectionManager.enable();
+          if (this._terminal.element) {
+            this._terminal.element.classList.remove('enable-mouse-events');
+          }
+          if (this._terminal.selectionManager) {
+            this._terminal.selectionManager.enable();
+          }
           break;
         case 1004: // send focusin/focusout events
           this._terminal.sendFocus = false;

src/Linkifier.test.ts

@@ -41,8 +41,8 @@ describe('Linkifier', () => {
 
   beforeEach(() => {
     terminal = new MockTerminal();
-    terminal.cols = 100;
-    terminal.rows = 10;
+    (terminal as any).cols = 100;
+    (terminal as any).rows = 10;
     terminal.buffer = new MockBuffer();
     (<MockBuffer>terminal.buffer).setLines(new CircularList<IBufferLine>(20));
     terminal.buffer.ydisp = 0;
@@ -65,7 +65,7 @@ describe('Linkifier', () => {
   function assertLinkifiesRow(rowText: string, linkMatcherRegex: RegExp, links: {x: number, length: number}[], done: MochaDone): void {
     addRow(rowText);
     linkifier.registerLinkMatcher(linkMatcherRegex, () => {});
-    terminal.rows = terminal.buffer.lines.length - 1;
+    (terminal as any).rows = terminal.buffer.lines.length - 1;
     linkifier.linkifyRows();
     // Allow linkify to happen
     setTimeout(() => {
@@ -142,19 +142,19 @@ describe('Linkifier', () => {
       });
       describe('multi-line links', () => {
         it('should match links that start on line 1/2 of a wrapped line and end on the last character of line 1/2', done => {
-          terminal.cols = 4;
+          (terminal as any).cols = 4;
           assertLinkifiesMultiLineLink('12345', /1234/, [{x1: 0, x2: 4, y1: 0, y2: 0}], done);
         });
         it('should match links that start on line 1/2 of a wrapped line and wrap to line 2/2', done => {
-          terminal.cols = 4;
+          (terminal as any).cols = 4;
           assertLinkifiesMultiLineLink('12345', /12345/, [{x1: 0, x2: 1, y1: 0, y2: 1}], done);
         });
         it('should match links that start and end on line 2/2 of a wrapped line', done => {
-          terminal.cols = 4;
+          (terminal as any).cols = 4;
           assertLinkifiesMultiLineLink('12345678', /5678/, [{x1: 0, x2: 4, y1: 1, y2: 1}], done);
         });
         it('should match links that start on line 2/3 of a wrapped line and wrap to line 3/3', done => {
-          terminal.cols = 4;
+          (terminal as any).cols = 4;
           assertLinkifiesMultiLineLink('123456789', /56789/, [{x1: 0, x2: 1, y1: 1, y2: 2}], done);
         });
       });

src/SelectionManager.test.ts

@@ -45,8 +45,8 @@ describe('SelectionManager', () => {
 
   beforeEach(() => {
     terminal = new TestMockTerminal();
-    terminal.cols = 80;
-    terminal.rows = 2;
+    (terminal as any).cols = 80;
+    (terminal as any).rows = 2;
     terminal.options.scrollback = 100;
     terminal.buffers = new BufferSet(terminal);
     terminal.buffer = terminal.buffers.active;

src/SelectionManager.ts

@@ -552,7 +552,7 @@ export class SelectionManager extends EventEmitter implements ISelectionManager
    */
   private _onMouseMove(event: MouseEvent): void {
     // If the mousemove listener is active it means that a selection is
-    // currently being made, we should stop propogation to prevent mouse events
+    // currently being made, we should stop propagation to prevent mouse events
     // to be sent to the pty.
     event.stopImmediatePropagation();
 

src/SelectionModel.test.ts

@@ -23,8 +23,8 @@ describe('SelectionManager', () => {
 
   beforeEach(() => {
     terminal = new MockTerminal();
-    terminal.cols = 80;
-    terminal.rows = 2;
+    (terminal as any).cols = 80;
+    (terminal as any).rows = 2;
     terminal.options.scrollback = 10;
     terminal.buffers = new BufferSet(terminal);
     terminal.buffer = terminal.buffers.active;

src/Terminal.ts

@@ -64,10 +64,15 @@ const document = (typeof window !== 'undefined') ? window.document : null;
 const WRITE_BUFFER_PAUSE_THRESHOLD = 5;
 
 /**
- * The number of writes to perform in a single batch before allowing the
- * renderer to catch up with a 0ms setTimeout.
+ * The max number of ms to spend on writes before allowing the renderer to
+ * catch up with a 0ms setTimeout. A value of < 33 to keep us close to
+ * 30fps, and a value of < 16 to try to run at 60fps. Of course, the real FPS
+ * depends on the time it takes for the renderer to draw the frame.
  */
-const WRITE_BATCH_SIZE = 300;
+const WRITE_TIMEOUT_MS = 12;
+
+const MINIMUM_COLS = 2; // Less than 2 can mess with wide chars
+const MINIMUM_ROWS = 1;
 
 /**
  * The set of options that only have an effect when set in the Terminal constructor.
@@ -105,8 +110,7 @@ const DEFAULT_OPTIONS: ITerminalOptions = {
   tabStopWidth: 8,
   theme: null,
   rightClickSelectsWord: Browser.isMac,
-  rendererType: 'canvas',
-  experimentalBufferLineImpl: 'TypedArray'
+  rendererType: 'canvas'
 };
 
 export class Terminal extends EventEmitter implements ITerminal, IDisposable, IInputHandlingTerminal {
@@ -263,8 +267,8 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
     // TODO: WHy not document.body?
     this._parent = document ? document.body : null;
 
-    this.cols = this.options.cols;
-    this.rows = this.options.rows;
+    this.cols = Math.max(this.options.cols, MINIMUM_COLS);
+    this.rows = Math.max(this.options.rows, MINIMUM_ROWS);
 
     if (this.options.handler) {
       this.on('data', this.options.handler);
@@ -497,11 +501,6 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
         }
         break;
       case 'tabStopWidth': this.buffers.setupTabStops(); break;
-      case 'experimentalBufferLineImpl':
-        this.buffers.normal.setBufferLineFactory(value);
-        this.buffers.alt.setBufferLineFactory(value);
-        this._blankLine = null;
-        break;
     }
     // Inform renderer of changes
     if (this.renderer) {
@@ -569,12 +568,12 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
       // Firefox doesn't appear to fire the contextmenu event on right click
       this.register(addDisposableDomListener(this.element, 'mousedown', (event: MouseEvent) => {
         if (event.button === 2) {
-          rightClickHandler(event, this.textarea, this.selectionManager, this.options.rightClickSelectsWord);
+          rightClickHandler(event, this, this.selectionManager, this.options.rightClickSelectsWord);
         }
       }));
     } else {
       this.register(addDisposableDomListener(this.element, 'contextmenu', (event: MouseEvent) => {
-        rightClickHandler(event, this.textarea, this.selectionManager, this.options.rightClickSelectsWord);
+        rightClickHandler(event, this, this.selectionManager, this.options.rightClickSelectsWord);
       }));
     }
 
@@ -586,7 +585,7 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
       // that the regular click event doesn't fire for the middle mouse button.
       this.register(addDisposableDomListener(this.element, 'auxclick', (event: MouseEvent) => {
         if (event.button === 1) {
-          moveTextAreaUnderMouseCursor(event, this.textarea);
+          moveTextAreaUnderMouseCursor(event, this);
         }
       }));
     }
@@ -739,6 +738,13 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
     this.register(addDisposableDomListener(this._viewportElement, 'scroll', () => this.selectionManager.refresh()));
 
     this.mouseHelper = new MouseHelper(this.renderer);
+    // apply mouse event classes set by escape codes before terminal was attached
+    this.element.classList.toggle('enable-mouse-events', this.mouseEvents);
+    if (this.mouseEvents) {
+      this.selectionManager.disable();
+    } else {
+      this.selectionManager.enable();
+    }
 
     if (this.options.screenReaderMode) {
       // Note that this must be done *after* the renderer is created in order to
@@ -1180,17 +1186,12 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
    */
   public scroll(isWrapped: boolean = false): void {
     let newLine: IBufferLine;
-    const useRecycling = this.options.experimentalBufferLineImpl !== 'JsArray';
-    if (useRecycling) {
-      newLine = this._blankLine;
-      if (!newLine || newLine.length !== this.cols || newLine.get(0)[CHAR_DATA_ATTR_INDEX] !== this.eraseAttr()) {
-        newLine = this.buffer.getBlankLine(this.eraseAttr(), isWrapped);
-        this._blankLine = newLine;
-      }
-      newLine.isWrapped = isWrapped;
-    } else {
+    newLine = this._blankLine;
+    if (!newLine || newLine.length !== this.cols || newLine.get(0)[CHAR_DATA_ATTR_INDEX] !== this.eraseAttr()) {
       newLine = this.buffer.getBlankLine(this.eraseAttr(), isWrapped);
+      this._blankLine = newLine;
     }
+    newLine.isWrapped = isWrapped;
 
     const topRow = this.buffer.ybase + this.buffer.scrollTop;
     const bottomRow = this.buffer.ybase + this.buffer.scrollBottom;
@@ -1201,17 +1202,13 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
 
       // Insert the line using the fastest method
       if (bottomRow === this.buffer.lines.length - 1) {
-        if (useRecycling) {
-          if (willBufferBeTrimmed) {
-            this.buffer.lines.recycle().copyFrom(newLine);
-          } else {
-            this.buffer.lines.push(newLine.clone());
-          }
+        if (willBufferBeTrimmed) {
+          this.buffer.lines.recycle().copyFrom(newLine);
         } else {
-          this.buffer.lines.push(newLine);
+          this.buffer.lines.push(newLine.clone());
         }
       } else {
-        this.buffer.lines.splice(bottomRow + 1, 0, (useRecycling) ? newLine.clone() : newLine);
+        this.buffer.lines.splice(bottomRow + 1, 0, newLine.clone());
       }
 
       // Only adjust ybase and ydisp when the buffer is not trimmed
@@ -1233,7 +1230,7 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
       // scrollback, instead we can just shift them in-place.
       const scrollRegionHeight = bottomRow - topRow + 1/*as it's zero-based*/;
       this.buffer.lines.shiftElements(topRow + 1, scrollRegionHeight - 1, -1);
-      this.buffer.lines.set(bottomRow, (useRecycling) ? newLine.clone() : newLine);
+      this.buffer.lines.set(bottomRow, newLine.clone());
     }
 
     // Move the viewport to the bottom of the buffer unless the user is
@@ -1353,19 +1350,20 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
     }
   }
 
-  protected _innerWrite(): void {
+  protected _innerWrite(bufferOffset: number = 0): void {
     // Ensure the terminal isn't disposed
     if (this._isDisposed) {
       this.writeBuffer = [];
     }
 
-    const writeBatch = this.writeBuffer.splice(0, WRITE_BATCH_SIZE);
-    while (writeBatch.length > 0) {
-      const data = writeBatch.shift();
+    const startTime = Date.now();
+    while (this.writeBuffer.length > bufferOffset) {
+      const data = this.writeBuffer[bufferOffset];
+      bufferOffset++;
 
       // If XOFF was sent in order to catch up with the pty process, resume it if
-      // the writeBuffer is empty to allow more data to come in.
-      if (this._xoffSentToCatchUp && writeBatch.length === 0 && this.writeBuffer.length === 0) {
+      // we reached the end of the writeBuffer to allow more data to come in.
+      if (this._xoffSentToCatchUp && this.writeBuffer.length === bufferOffset) {
         this.handler(C0.DC1);
         this._xoffSentToCatchUp = false;
       }
@@ -1383,12 +1381,17 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
 
       this.updateRange(this.buffer.y);
       this.refresh(this._refreshStart, this._refreshEnd);
+
+      if (Date.now() - startTime >= WRITE_TIMEOUT_MS) {
+        break;
+      }
     }
-    if (this.writeBuffer.length > 0) {
+    if (this.writeBuffer.length > bufferOffset) {
       // Allow renderer to catch up before processing the next batch
-      setTimeout(() => this._innerWrite(), 0);
+      setTimeout(() => this._innerWrite(bufferOffset), 0);
     } else {
       this._writeInProgress = false;
+      this.writeBuffer = [];
     }
   }
 
@@ -1406,7 +1409,7 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
    * processed by the terminal and what keys should not.
    * @param customKeyEventHandler The custom KeyboardEvent handler to attach.
    * This is a function that takes a KeyboardEvent, allowing consumers to stop
-   * propogation and/or prevent the default action. The function returns whether
+   * propagation and/or prevent the default action. The function returns whether
    * the event should be processed by xterm.js.
    */
   public attachCustomKeyEventHandler(customKeyEventHandler: CustomKeyEventHandler): void {
@@ -1706,8 +1709,8 @@ export class Terminal extends EventEmitter implements ITerminal, IDisposable, II
       return;
     }
 
-    if (x < 1) x = 1;
-    if (y < 1) y = 1;
+    if (x < MINIMUM_COLS) x = MINIMUM_COLS;
+    if (y < MINIMUM_ROWS) y = MINIMUM_ROWS;
 
     this.buffers.resize(x, y);
 

src/Types.ts

@@ -111,7 +111,7 @@ export interface ICompositionHelper {
  */
 export interface IInputHandler {
   parse(data: string): void;
-  print(data: string, start: number, end: number): void;
+  print(data: Uint32Array, start: number, end: number): void;
 
   /** C0 BEL */ bell(): void;
   /** C0 LF */ lineFeed(): void;
@@ -237,7 +237,7 @@ export interface IBufferAccessor {
 }
 
 export interface IElementAccessor {
-  element: HTMLElement;
+  readonly element: HTMLElement;
 }
 
 export interface ILinkifierAccessor {
@@ -452,18 +452,26 @@ export interface IParsingState {
 * DCS handler signature for EscapeSequenceParser.
 * EscapeSequenceParser handles DCS commands via separate
 * subparsers that get hook/unhooked and can handle
-* arbitrary amount of print data.
+* arbitrary amount of data.
+*
 * On entering a DSC sequence `hook` is called by
 * `EscapeSequenceParser`. Use it to initialize or reset
 * states needed to handle the current DCS sequence.
+* Note: A DCS parser is only instantiated once, therefore
+* you cannot rely on the ctor to reinitialize state.
+*
 * EscapeSequenceParser will call `put` several times if the
-* parsed string got splitted, therefore you might have to collect
-* `data` until `unhook` is called. `unhook` marks the end
-* of the current DCS sequence.
+* parsed data got split, therefore you might have to collect
+* `data` until `unhook` is called.
+* Note: `data` is borrowed, if you cannot process the data
+* in chunks you have to copy it, doing otherwise will lead to
+* data losses or corruption.
+*
+* `unhook` marks the end of the current DCS sequence.
 */
 export interface IDcsHandler {
   hook(collect: string, params: number[], flag: number): void;
-  put(data: string, start: number, end: number): void;
+  put(data: Uint32Array, start: number, end: number): void;
   unhook(): void;
 }
 
@@ -480,9 +488,9 @@ export interface IEscapeSequenceParser extends IDisposable {
    * Parse string `data`.
    * @param data The data to parse.
    */
-  parse(data: string): void;
+  parse(data: Uint32Array, length: number): void;
 
-  setPrintHandler(callback: (data: string, start: number, end: number) => void): void;
+  setPrintHandler(callback: (data: Uint32Array, start: number, end: number) => void): void;
   clearPrintHandler(): void;
 
   setExecuteHandler(flag: string, callback: () => void): void;
@@ -522,14 +530,10 @@ export interface IBufferLine {
   insertCells(pos: number, n: number, ch: CharData): void;
   deleteCells(pos: number, n: number, fill: CharData): void;
   replaceCells(start: number, end: number, fill: CharData): void;
-  resize(cols: number, fill: CharData, shrink?: boolean): void;
+  resize(cols: number, fill: CharData): void;
   fill(fillCharData: CharData): void;
   copyFrom(line: IBufferLine): void;
   clone(): IBufferLine;
   getTrimmedLength(): number;
   translateToString(trimRight?: boolean, startCol?: number, endCol?: number): string;
 }
-
-export interface IBufferLineConstructor {
-  new(cols: number, fillCharData?: CharData, isWrapped?: boolean): IBufferLine;
-}

src/addons/winptyCompat/winptyCompat.ts

@@ -19,6 +19,8 @@ export function winptyCompatInit(terminal: Terminal): void {
     return;
   }
 
+  (addonTerminal._core as any).isWinptyCompatEnabled = true;
+
   // Winpty does not support wraparound mode which means that lines will never
   // be marked as wrapped. This causes issues for things like copying a line
   // retaining the wrapped new line characters or if consumers are listening

src/common/CircularList.ts

@@ -6,6 +6,16 @@
 import { EventEmitter } from './EventEmitter';
 import { ICircularList } from './Types';
 
+export interface IInsertEvent {
+  index: number;
+  amount: number;
+}
+
+export interface IDeleteEvent {
+  index: number;
+  amount: number;
+}
+
 /**
  * Represents a circular list; a list with a maximum size that wraps around when push is called,
  * overriding values at the start of the list.
@@ -91,7 +101,7 @@ export class CircularList<T> extends EventEmitter implements ICircularList<T> {
     this._array[this._getCyclicIndex(this._length)] = value;
     if (this._length === this._maxLength) {
       this._startIndex = ++this._startIndex % this._maxLength;
-      this.emit('trim', 1);
+      this.emitMayRemoveListeners('trim', 1);
     } else {
       this._length++;
     }
@@ -107,7 +117,7 @@ export class CircularList<T> extends EventEmitter implements ICircularList<T> {
       throw new Error('Can only recycle when the buffer is full');
     }
     this._startIndex = ++this._startIndex % this._maxLength;
-    this.emit('trim', 1);
+    this.emitMayRemoveListeners('trim', 1);
     return this._array[this._getCyclicIndex(this._length - 1)]!;
   }
 
@@ -144,24 +154,22 @@ export class CircularList<T> extends EventEmitter implements ICircularList<T> {
       this._length -= deleteCount;
     }
 
-    if (items && items.length) {
-      // Add items
-      for (let i = this._length - 1; i >= start; i--) {
-        this._array[this._getCyclicIndex(i + items.length)] = this._array[this._getCyclicIndex(i)];
-      }
-      for (let i = 0; i < items.length; i++) {
-        this._array[this._getCyclicIndex(start + i)] = items[i];
-      }
+    // Add items
+    for (let i = this._length - 1; i >= start; i--) {
+      this._array[this._getCyclicIndex(i + items.length)] = this._array[this._getCyclicIndex(i)];
+    }
+    for (let i = 0; i < items.length; i++) {
+      this._array[this._getCyclicIndex(start + i)] = items[i];
+    }
 
-      // Adjust length as needed
-      if (this._length + items.length > this._maxLength) {
-        const countToTrim = (this._length + items.length) - this._maxLength;
-        this._startIndex += countToTrim;
-        this._length = this._maxLength;
-        this.emit('trim', countToTrim);
-      } else {
-        this._length += items.length;
-      }
+    // Adjust length as needed
+    if (this._length + items.length > this._maxLength) {
+      const countToTrim = (this._length + items.length) - this._maxLength;
+      this._startIndex += countToTrim;
+      this._length = this._maxLength;
+      this.emitMayRemoveListeners('trim', countToTrim);
+    } else {
+      this._length += items.length;
     }
   }
 
@@ -175,7 +183,7 @@ export class CircularList<T> extends EventEmitter implements ICircularList<T> {
     }
     this._startIndex += count;
     this._length -= count;
-    this.emit('trim', count);
+    this.emitMayRemoveListeners('trim', count);
   }
 
   public shiftElements(start: number, count: number, offset: number): void {
@@ -199,7 +207,7 @@ export class CircularList<T> extends EventEmitter implements ICircularList<T> {
         while (this._length > this._maxLength) {
           this._length--;
           this._startIndex++;
-          this.emit('trim', 1);
+          this.emitMayRemoveListeners('trim', 1);
         }
       }
     } else {

src/common/EventEmitter.ts

@@ -23,7 +23,7 @@ export class EventEmitter extends Disposable implements IEventEmitter, IDisposab
   }
 
   /**
-   * Adds a disposabe listener to the EventEmitter, returning the disposable.
+   * Adds a disposable listener to the EventEmitter, returning the disposable.
    * @param type The event type.
    * @param handler The handler for the listener.
    */
@@ -75,6 +75,19 @@ export class EventEmitter extends Disposable implements IEventEmitter, IDisposab
     }
   }
 
+  public emitMayRemoveListeners(type: string, ...args: any[]): void {
+    if (!this._events[type]) {
+      return;
+    }
+    const obj = this._events[type];
+    let length = obj.length;
+    for (let i = 0; i < obj.length; i++) {
+      obj[i].apply(this, args);
+      i -= length - obj.length;
+      length = obj.length;
+    }
+  }
+
   public listeners(type: string): XtermListener[] {
     return this._events[type] || [];
   }

src/common/TypedArrayUtils.test.ts

@@ -3,21 +3,20 @@
  * @license MIT
  */
 import { assert } from 'chai';
-import { fillFallback } from './TypedArrayUtils';
+import { fillFallback, concat } from './TypedArrayUtils';
 
 type TypedArray = Uint8Array | Uint16Array | Uint32Array | Uint8ClampedArray
   | Int8Array | Int16Array | Int32Array
   | Float32Array | Float64Array;
 
-describe('polyfill conformance tests', function(): void {
-
-  function deepEquals(a: TypedArray, b: TypedArray): void {
-    assert.equal(a.length, b.length);
-    for (let i = 0; i < a.length; ++i) {
-      assert.equal(a[i], b[i]);
-    }
+function deepEquals(a: TypedArray, b: TypedArray): void {
+  assert.equal(a.length, b.length);
+  for (let i = 0; i < a.length; ++i) {
+    assert.equal(a[i], b[i]);
   }
+}
 
+describe('polyfill conformance tests', function(): void {
   describe('TypedArray.fill', function(): void {
     it('should work with all typed array types', function(): void {
       const u81 = new Uint8Array(5);
@@ -87,3 +86,12 @@ describe('polyfill conformance tests', function(): void {
     });
   });
 });
+
+describe('typed array convenience functions', () => {
+  it('concat', () => {
+    const a = new Uint8Array([1, 2, 3, 4, 5]);
+    const b = new Uint8Array([6, 7, 8, 9, 0]);
+    const merged = concat(a, b);
+    deepEquals(merged, new Uint8Array([1, 2, 3, 4, 5, 6, 7, 8, 9, 0]));
+  });
+});

src/common/TypedArrayUtils.ts

@@ -3,15 +3,15 @@
  * @license MIT
  */
 
+export type TypedArray = Uint8Array | Uint16Array | Uint32Array | Uint8ClampedArray
+  | Int8Array | Int16Array | Int32Array
+  | Float32Array | Float64Array;
+
+
 /**
  * polyfill for TypedArray.fill
  * This is needed to support .fill in all safari versions and IE 11.
  */
-
-type TypedArray = Uint8Array | Uint16Array | Uint32Array | Uint8ClampedArray
-  | Int8Array | Int16Array | Int32Array
-  | Float32Array | Float64Array;
-
 export function fill<T extends TypedArray>(array: T, value: number, start?: number, end?: number): T {
   // all modern engines that support .fill
   if (array.fill) {
@@ -39,3 +39,14 @@ export function fillFallback<T extends TypedArray>(array: T, value: number, star
   }
   return array;
 }
+
+/**
+ * Concat two typed arrays `a` and `b`.
+ * Returns a new typed array.
+ */
+export function concat<T extends TypedArray>(a: T, b: T): T {
+  const result = new (a.constructor as any)(a.length + b.length);
+  result.set(a);
+  result.set(b, a.length);
+  return result;
+}

src/core/input/Keyboard.ts

@@ -44,7 +44,7 @@ export function evaluateKeyboardEvent(
 ): IKeyboardResult {
   const result: IKeyboardResult = {
     type: KeyboardResultType.SEND_KEY,
-    // Whether to cancel event propogation (NOTE: this may not be needed since the event is
+    // Whether to cancel event propagation (NOTE: this may not be needed since the event is
     // canceled at the end of keyDown
     cancel: false,
     // The new key even to emit
@@ -349,9 +349,8 @@ export function evaluateKeyboardEvent(
         if (ev.keyCode === 65) { // cmd + a
           result.type = KeyboardResultType.SELECT_ALL;
         }
-      } else if (ev.key && !ev.ctrlKey && !ev.altKey && !ev.metaKey &&
-          ev.keyCode >= 48 && ev.keyCode !== 144 && ev.keyCode !== 145) {
-        // Include only keys that that result in a character; don't include num lock and scroll lock
+      } else if (ev.key && !ev.ctrlKey && !ev.altKey && !ev.metaKey && ev.keyCode >= 48 && ev.key.length === 1) {
+        // Include only keys that that result in a _single_ character; don't include num lock, volume up, etc.
         result.key = ev.key;
       }
       break;

src/core/input/TextDecoder.test.ts

@@ -0,0 +1,67 @@
+/**
+ * Copyright (c) 2019 The xterm.js authors. All rights reserved.
+ * @license MIT
+ */
+
+import { assert } from 'chai';
+import { StringToUtf32, stringFromCodePoint, utf32ToString } from './TextDecoder';
+
+describe('text encodings', () => {
+  it('stringFromCodePoint/utf32ToString', () => {
+    const s = 'abcdefg';
+    const data = new Uint32Array(s.length);
+    for (let i = 0; i < s.length; ++i) {
+      data[i] = s.charCodeAt(i);
+      assert.equal(stringFromCodePoint(data[i]), s[i]);
+    }
+    assert.equal(utf32ToString(data), s);
+  });
+
+  describe('StringToUtf32 Decoder', () => {
+    describe('full codepoint test', () => {
+      it('0..65535', () => {
+        const decoder = new StringToUtf32();
+        const target = new Uint32Array(5);
+        for (let i = 0; i < 65536; ++i) {
+          // skip surrogate pairs
+          if (i >= 0xD800 && i <= 0xDFFF) {
+            continue;
+          }
+          const length = decoder.decode(String.fromCharCode(i), target);
+          assert.equal(length, 1);
+          assert.equal(target[0], i);
+          assert.equal(utf32ToString(target, 0, length), String.fromCharCode(i));
+          decoder.clear();
+        }
+      });
+      it('65536..0x10FFFF (surrogates)', function(): void {
+        this.timeout(20000);
+        const decoder = new StringToUtf32();
+        const target = new Uint32Array(5);
+        for (let i = 65536; i < 0x10FFFF; ++i) {
+          const codePoint = i - 0x10000;
+          const s = String.fromCharCode((codePoint >> 10) + 0xD800) + String.fromCharCode((codePoint % 0x400) + 0xDC00);
+          const length = decoder.decode(s, target);
+          assert.equal(length, 1);
+          assert.equal(target[0], i);
+          assert.equal(utf32ToString(target, 0, length), s);
+          decoder.clear();
+        }
+      });
+    });
+
+    describe('stream handling', () => {
+      it('surrogates mixed advance by 1', () => {
+        const decoder = new StringToUtf32();
+        const target = new Uint32Array(5);
+        const input = 'Ä€𝄞Ö𝄞€Ü𝄞€';
+        let decoded = '';
+        for (let i = 0; i < input.length; ++i) {
+          const written = decoder.decode(input[i], target);
+          decoded += utf32ToString(target, written);
+        }
+        assert(decoded, 'Ä€𝄞Ö𝄞€Ü𝄞€');
+      });
+    });
+  });
+});

src/core/input/TextDecoder.ts

@@ -0,0 +1,109 @@
+/**
+ * Copyright (c) 2019 The xterm.js authors. All rights reserved.
+ * @license MIT
+ */
+
+/**
+ * StringToUtf32 - decodes UTF16 sequences into UTF32 codepoints.
+ * To keep the decoder in line with JS strings it handles single surrogates as UCS2.
+ */
+export class StringToUtf32 {
+  private _interim: number = 0;
+
+  /**
+   * Clears interim and resets decoder to clean state.
+   */
+  public clear(): void {
+    this._interim = 0;
+  }
+
+  /**
+   * Decode JS string to UTF32 codepoints.
+   * The methods assumes stream input and will store partly transmitted
+   * surrogate pairs and decode them with the next data chunk.
+   * Note: The method does no bound checks for target, therefore make sure
+   * the provided input data does not exceed the size of `target`.
+   * Returns the number of written codepoints in `target`.
+   */
+  decode(input: string, target: Uint32Array): number {
+    const length = input.length;
+
+    if (!length) {
+      return 0;
+    }
+
+    let size = 0;
+    let startPos = 0;
+
+    // handle leftover surrogate high
+    if (this._interim) {
+      const second = input.charCodeAt(startPos++);
+      if (0xDC00 <= second && second <= 0xDFFF) {
+        target[size++] = (this._interim - 0xD800) * 0x400 + second - 0xDC00 + 0x10000;
+      } else {
+        // illegal codepoint (USC2 handling)
+        target[size++] = this._interim;
+        target[size++] = second;
+      }
+      this._interim = 0;
+    }
+
+    for (let i = startPos; i < length; ++i) {
+      const code = input.charCodeAt(i);
+      // surrogate pair first
+      if (0xD800 <= code && code <= 0xDBFF) {
+        if (++i >= length) {
+          this._interim = code;
+          return size;
+        }
+        const second = input.charCodeAt(i);
+        if (0xDC00 <= second && second <= 0xDFFF) {
+          target[size++] = (code - 0xD800) * 0x400 + second - 0xDC00 + 0x10000;
+        } else {
+          // illegal codepoint (USC2 handling)
+          target[size++] = code;
+          target[size++] = second;
+        }
+        continue;
+      }
+      target[size++] = code;
+    }
+    return size;
+  }
+}
+
+/**
+ * Convert UTF32 codepoint into JS string.
+ */
+export function stringFromCodePoint(codePoint: number): string {
+  if (codePoint > 0xFFFF) {
+    // UTF32 to UTF16 conversion (see comments in utf32ToString)
+    codePoint -= 0x10000;
+    return String.fromCharCode((codePoint >> 10) + 0xD800) + String.fromCharCode((codePoint % 0x400) + 0xDC00);
+  }
+  return String.fromCharCode(codePoint);
+}
+
+/**
+ * Convert UTF32 char codes into JS string.
+ * Basically the same as `stringFromCodePoint` but for multiple codepoints
+ * in a loop (which is a lot faster).
+ */
+export function utf32ToString(data: Uint32Array, start: number = 0, end: number = data.length): string {
+  let result = '';
+  for (let i = start; i < end; ++i) {
+    let codepoint = data[i];
+    if (codepoint > 0xFFFF) {
+      // JS string are encoded as UTF16, thus a non BMP codepoint gets converted into a surrogate pair
+      // conversion rules:
+      //  - subtract 0x10000 from code point, leaving a 20 bit number
+      //  - add high 10 bits to 0xD800  --> first surrogate
+      //  - add low 10 bits to 0xDC00   --> second surrogate
+      codepoint -= 0x10000;
+      result += String.fromCharCode((codepoint >> 10) + 0xD800) + String.fromCharCode((codepoint % 0x400) + 0xDC00);
+    } else {
+      result += String.fromCharCode(codepoint);
+    }
+  }
+  return result;
+}

src/renderer/BaseRenderLayer.ts

@@ -241,7 +241,7 @@ export abstract class BaseRenderLayer implements IRenderLayer {
     this._ctx.fillText(
         charData[CHAR_DATA_CHAR_INDEX],
         x * this._scaledCellWidth + this._scaledCharLeft,
-        (y + 0.5) * this._scaledCellHeight + this._scaledCharTop);
+        y * this._scaledCellHeight + this._scaledCharTop + this._scaledCharHeight / 2);
   }
 
   /**
@@ -316,7 +316,7 @@ export abstract class BaseRenderLayer implements IRenderLayer {
     this._ctx.fillText(
         chars,
         x * this._scaledCellWidth + this._scaledCharLeft,
-        (y + 0.5) * this._scaledCellHeight + this._scaledCharTop);
+        y * this._scaledCellHeight + this._scaledCharTop + this._scaledCharHeight / 2);
     this._ctx.restore();
   }
 

src/renderer/Renderer.ts

@@ -70,7 +70,7 @@ export class Renderer extends EventEmitter implements IRenderer {
     // Detect whether IntersectionObserver is detected and enable renderer pause
     // and resume based on terminal visibility if so
     if ('IntersectionObserver' in window) {
-      const observer = new IntersectionObserver(e => this.onIntersectionChange(e[0]), { threshold: 0 });
+      const observer = new IntersectionObserver(e => this.onIntersectionChange(e[e.length - 1]), { threshold: 0 });
       observer.observe(this._terminal.element);
       this.register({ dispose: () => observer.disconnect() });
     }

src/renderer/SelectionRenderLayer.ts

@@ -55,6 +55,7 @@ export class SelectionRenderLayer extends BaseRenderLayer {
 
     // Selection does not exist
     if (!start || !end) {
+      this._clearState();
       return;
     }
 

src/ui/CharMeasure.ts

@@ -46,9 +46,10 @@ export class CharMeasure extends EventEmitter implements ICharMeasure {
     if (geometry.width === 0 || geometry.height === 0) {
       return;
     }
-    if (this._width !== geometry.width || this._height !== geometry.height) {
+    const adjustedHeight = Math.ceil(geometry.height);
+    if (this._width !== geometry.width || this._height !== adjustedHeight) {
       this._width = geometry.width;
-      this._height = Math.ceil(geometry.height);
+      this._height = adjustedHeight;
       this.emit('charsizechanged');
     }
   }

src/ui/Clipboard.ts

@@ -85,26 +85,32 @@ export function pasteHandler(ev: ClipboardEvent, term: ITerminal): void {
  * @param ev The original right click event to be handled.
  * @param textarea The terminal's textarea.
  */
-export function moveTextAreaUnderMouseCursor(ev: MouseEvent, textarea: HTMLTextAreaElement): void {
-  // Bring textarea at the cursor position
-  textarea.style.position = 'fixed';
-  textarea.style.width = '20px';
-  textarea.style.height = '20px';
-  textarea.style.left = (ev.clientX - 10) + 'px';
-  textarea.style.top = (ev.clientY - 10) + 'px';
-  textarea.style.zIndex = '1000';
+export function moveTextAreaUnderMouseCursor(ev: MouseEvent, term: ITerminal): void {
 
-  textarea.focus();
+  // Calculate textarea position relative to the screen element
+  const pos = term.screenElement.getBoundingClientRect();
+  const left = ev.clientX - pos.left - 10;
+  const top = ev.clientY - pos.top - 10;
+
+  // Bring textarea at the cursor position
+  term.textarea.style.position = 'absolute';
+  term.textarea.style.width = '20px';
+  term.textarea.style.height = '20px';
+  term.textarea.style.left = `${left}px`;
+  term.textarea.style.top = `${top}px`;
+  term.textarea.style.zIndex = '1000';
+
+  term.textarea.focus();
 
   // Reset the terminal textarea's styling
   // Timeout needs to be long enough for click event to be handled.
   setTimeout(() => {
-    textarea.style.position = null;
-    textarea.style.width = null;
-    textarea.style.height = null;
-    textarea.style.left = null;
-    textarea.style.top = null;
-    textarea.style.zIndex = null;
+    term.textarea.style.position = null;
+    term.textarea.style.width = null;
+    term.textarea.style.height = null;
+    term.textarea.style.left = null;
+    term.textarea.style.top = null;
+    term.textarea.style.zIndex = null;
   }, 200);
 }
 
@@ -115,14 +121,14 @@ export function moveTextAreaUnderMouseCursor(ev: MouseEvent, textarea: HTMLTextA
  * @param selectionManager The terminal's selection manager.
  * @param shouldSelectWord If true and there is no selection the current word will be selected
  */
-export function rightClickHandler(ev: MouseEvent, textarea: HTMLTextAreaElement, selectionManager: ISelectionManager, shouldSelectWord: boolean): void {
-  moveTextAreaUnderMouseCursor(ev, textarea);
+export function rightClickHandler(ev: MouseEvent, term: ITerminal, selectionManager: ISelectionManager, shouldSelectWord: boolean): void {
+  moveTextAreaUnderMouseCursor(ev, term);
 
   if (shouldSelectWord && !selectionManager.isClickInSelection(ev)) {
     selectionManager.selectWordAtCursor(ev);
   }
 
   // Get textarea ready to copy from the context menu
-  textarea.value = selectionManager.selectionText;
-  textarea.select();
+  term.textarea.value = selectionManager.selectionText;
+  term.textarea.select();
 }

src/ui/Lifecycle.ts

@@ -6,7 +6,7 @@
 import { IDisposable } from 'xterm';
 
 /**
- * Adds a disposabe listener to a node in the DOM, returning the disposable.
+ * Adds a disposable listener to a node in the DOM, returning the disposable.
  * @param type The event type.
  * @param handler The handler for the listener.
  */

src/ui/MouseZoneManager.ts

@@ -23,6 +23,7 @@ export class MouseZoneManager extends Disposable implements IMouseZoneManager {
 
   private _areZonesActive: boolean = false;
   private _mouseMoveListener: (e: MouseEvent) => any;
+  private _mouseLeaveListener: (e: MouseEvent) => any;
   private _clickListener: (e: MouseEvent) => any;
 
   private _tooltipTimeout: number = null;
@@ -38,6 +39,7 @@ export class MouseZoneManager extends Disposable implements IMouseZoneManager {
 
     // These events are expensive, only listen to it when mouse zones are active
     this._mouseMoveListener = e => this._onMouseMove(e);
+    this._mouseLeaveListener = e => this._onMouseLeave(e);
     this._clickListener = e => this._onClick(e);
   }
 
@@ -89,6 +91,7 @@ export class MouseZoneManager extends Disposable implements IMouseZoneManager {
     if (!this._areZonesActive) {
       this._areZonesActive = true;
       this._terminal.element.addEventListener('mousemove', this._mouseMoveListener);
+      this._terminal.element.addEventListener('mouseleave', this._mouseLeaveListener);
       this._terminal.element.addEventListener('click', this._clickListener);
     }
   }
@@ -97,6 +100,7 @@ export class MouseZoneManager extends Disposable implements IMouseZoneManager {
     if (this._areZonesActive) {
       this._areZonesActive = false;
       this._terminal.element.removeEventListener('mousemove', this._mouseMoveListener);
+      this._terminal.element.removeEventListener('mouseleave', this._mouseLeaveListener);
       this._terminal.element.removeEventListener('click', this._clickListener);
     }
   }
@@ -169,6 +173,18 @@ export class MouseZoneManager extends Disposable implements IMouseZoneManager {
     }
   }
 
+  private _onMouseLeave(e: MouseEvent): void {
+    // Fire the hover end callback and cancel any existing timer if the mouse
+    // leaves the terminal element
+    if (this._currentZone) {
+      this._currentZone.leaveCallback();
+      this._currentZone = null;
+      if (this._tooltipTimeout) {
+        clearTimeout(this._tooltipTimeout);
+      }
+    }
+  }
+
   private _onClick(e: MouseEvent): void {
     // Find the active zone and click it if found
     const zone = this._findZoneEventAt(e);

src/xterm.css

@@ -36,7 +36,6 @@
  */
 
 .xterm {
-    font-family: courier-new, courier, monospace;
     font-feature-settings: "liga" 0;
     position: relative;
     user-select: none;

typings/xterm.d.ts

@@ -101,17 +101,6 @@ declare module 'xterm' {
      */
     experimentalCharAtlas?: 'none' | 'static' | 'dynamic';
 
-    /**
-     * (EXPERIMENTAL) Defines which implementation to use for buffer lines.
-     *
-     * - 'JsArray': The default/stable implementation.
-     * - 'TypedArray': The new experimental implementation based on TypedArrays that is expected to
-     *   significantly boost performance and memory consumption. Use at your own risk.
-     *
-     * @deprecated This option will be removed in the future.
-     */
-    experimentalBufferLineImpl?: 'JsArray' | 'TypedArray';
-
     /**
      * The font size used to render text.
      */
@@ -325,28 +314,32 @@ declare module 'xterm' {
     /**
      * The element containing the terminal.
      */
-    element: HTMLElement;
+    readonly element: HTMLElement;
 
     /**
      * The textarea that accepts input for the terminal.
      */
-    textarea: HTMLTextAreaElement;
+    readonly textarea: HTMLTextAreaElement;
 
     /**
-     * The number of rows in the terminal's viewport.
+     * The number of rows in the terminal's viewport. Use
+     * `ITerminalOptions.rows` to set this in the constructor and
+     * `Terminal.resize` for when the terminal exists.
      */
-    rows: number;
+    readonly rows: number;
 
     /**
-     * The number of columns in the terminal's viewport.
+     * The number of columns in the terminal's viewport. Use
+     * `ITerminalOptions.cols` to set this in the constructor and
+     * `Terminal.resize` for when the terminal exists.
      */
-    cols: number;
+    readonly cols: number;
 
     /**
      * (EXPERIMENTAL) Get all markers registered against the buffer. If the alt
      * buffer is active this will always return [].
      */
-    markers: IMarker[];
+    readonly markers: IMarker[];
 
     /**
      * Natural language strings that can be localized.
@@ -450,7 +443,9 @@ declare module 'xterm' {
     addDisposableListener(type: string, handler: (...args: any[]) => void): IDisposable;
 
     /**
-     * Resizes the terminal.
+     * Resizes the terminal. It's best practice to debounce calls to resize,
+     * this will help ensure that the pty can respond to the resize event
+     * before another one occurs.
      * @param x The number of columns to resize to.
      * @param y The number of rows to resize to.
      */
@@ -476,7 +471,7 @@ declare module 'xterm' {
      * should be processed by the terminal and what keys should not.
      * @param customKeyEventHandler The custom KeyboardEvent handler to attach.
      * This is a function that takes a KeyboardEvent, allowing consumers to stop
-     * propogation and/or prevent the default action. The function returns
+     * propagation and/or prevent the default action. The function returns
      * whether the event should be processed by xterm.js.
      */
     attachCustomKeyEventHandler(customKeyEventHandler: (event: KeyboardEvent) => boolean): void;