Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add logic for eager evaluation in REPL #4277

Merged
merged 39 commits into from
Feb 2, 2025
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
8d80cee
feat: add draft logic of eager evaluation
Vinit-Pandit Dec 27, 2024
4a9205b
fix: fixing test failure
Vinit-Pandit Dec 27, 2024
4c15092
fix: fixing test failure
Vinit-Pandit Dec 27, 2024
e626b19
checking options.setting
Vinit-Pandit Dec 27, 2024
e43cab1
fix: style fix
Vinit-Pandit Dec 27, 2024
4f6efef
Merge branch 'stdlib-js:develop' into eager_evaluation_repl
Vinit-Pandit Jan 14, 2025
b43056d
chore: update copyright years
stdlib-bot Jan 14, 2025
6c8d7fd
style: fix indentation in manifest.json files
aayush0325 Jan 6, 2025
30f7796
fix: fixing clashing in end line of terminal
Vinit-Pandit Jan 24, 2025
72651d0
fix: removing unnessary conditions check
Vinit-Pandit Jan 24, 2025
7fce9d3
fix: adding suggested changes
Vinit-Pandit Jan 25, 2025
7f23f98
style: adding spaces in parenthesis
Vinit-Pandit Jan 25, 2025
9aa4c3e
refactor: change file name for consistency
Snehil-Shah Jan 29, 2025
7da041f
fix: add guard against invalid class initialization
Snehil-Shah Jan 29, 2025
e540283
refactor: use existing constant for hardcoded values
Snehil-Shah Jan 29, 2025
bebc0fe
refactor: move entangled logic outside and clean ups
Snehil-Shah Jan 29, 2025
cf2eaa7
fix: add stricter timeout and clean ups
Snehil-Shah Jan 30, 2025
81becd2
fix: update incorrect logic for handling multiline inputs
Snehil-Shah Jan 30, 2025
28c0f8d
style: update debug logs
Snehil-Shah Jan 30, 2025
998db52
feat: add logic to setting to toggle eagerevaluator
Vinit-Pandit Jan 30, 2025
36619b0
fix: make timeout even stricter
Snehil-Shah Jan 30, 2025
0f19131
style: clean ups
Snehil-Shah Jan 30, 2025
412fba3
style: clean ups
Vinit-Pandit Feb 1, 2025
b884fa6
style: clean up
Vinit-Pandit Feb 1, 2025
2e03eb3
style: clean up
Vinit-Pandit Feb 1, 2025
a40a99a
style: correctly place functions
Snehil-Shah Feb 1, 2025
f2f4c9b
docs: fix return type
kgryte Feb 2, 2025
08c68e5
docs: update description
kgryte Feb 2, 2025
8584782
docs: remove hyphen
kgryte Feb 2, 2025
8889314
docs: update description
kgryte Feb 2, 2025
69b4fe1
Apply suggestions from code review
kgryte Feb 2, 2025
d81e24f
docs: fix type
kgryte Feb 2, 2025
7260373
Apply suggestions from code review
kgryte Feb 2, 2025
35382da
docs: remove comment
kgryte Feb 2, 2025
55392c8
Apply suggestions from code review
kgryte Feb 2, 2025
ba01164
style: fix missing space
kgryte Feb 2, 2025
e94acec
refactor: remove unnecessary branch
kgryte Feb 2, 2025
c63d4f5
refactor: update debug message
kgryte Feb 2, 2025
6393eaf
refactor: update debug messages
kgryte Feb 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion lib/node_modules/@stdlib/repl/lib/defaults.js
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,10 @@ function defaults() {
'syntaxHighlighting': void 0,

// Theme for syntax highlighting:
'theme': 'stdlib-ansi-basic'
'theme': 'stdlib-ansi-basic',

// Flag indicating whether to enable eager Evaluation (note: default depends on whether TTY):
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
'eagerEvaluation': void 0
}
};
}
Expand Down
267 changes: 267 additions & 0 deletions lib/node_modules/@stdlib/repl/lib/eager_evaluation.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,267 @@
/**
* @license Apache-2.0
*
* Copyright (c) 2025 The Stdlib Authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

/* eslint-disable no-underscore-dangle, no-restricted-syntax, no-invalid-this */

'use strict';

// MODULES //

var readline = require( 'readline' );
var inspect = require( 'util' ).inspect;
var logger = require( 'debug' );
var parse = require( 'acorn' ).parse;
var replace = require( '@stdlib/string/replace' );
var setNonEnumerableReadOnly = require( '@stdlib/utils/define-nonenumerable-read-only-property' );
var processCommand = require( './process_command.js' );
var compileCommand = require( './compile_command.js' );


// VARIABLES //

var debug = logger( 'repl:EE' );
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
var AOPTS = {
'ecmaVersion': 'latest'
};
var tempDB = {
'base_sin': {
'isPure': true
}
};


// MAIN //

/**
* Constructor for creating a multi-line handler.
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
*
* @private
* @param {REPL} repl - repl instance
* @param {Object} rli - readline instance
* @param {Boolean} enabled - boolean indicating whether the syntax-highlighter should be initially enabled
* @returns {EagerEvaluator} eager-evaluator instance
*/
function EagerEvaluator( repl, rli, enabled ) {
this._repl = repl;
this._rli = rli;
this._cmd = repl._cmd;
this._enabled = enabled;
this._isPreview = false;

return this;
}
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved

/**
* Callback for handling a "keypress" event.
*
* @name onKeyPress
* @memberof EagerEvaluator.prototype
* @type {Function}
* @param {string} data - input data
* @param {(Object|void)} key - key object
* @returns {void}
*
*/
setNonEnumerableReadOnly(EagerEvaluator.prototype, 'onKeypress', function onKeyPress(data, key) {
kgryte marked this conversation as resolved.
Show resolved Hide resolved
var cursorPosition;
var executable;
var nlInd;
var code;
var opts;
var pre;
var res;
var tmp;

if (!this._enabled || this._rli.line === '' || key.name === 'paste-start' || (key.name === 'o' && key.ctrl) || key.name === 'return') {
return;
}
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved

code = this._cmd.join('\n') + this._rli.line;
opts = {
'timeout': this._repl._timeout,
'displayErrors': false,
'breakOnSigint': true // Node.js >=6.3.0
};
if (!this.isSideEffectFree(code)) {
debug('code have side effect');
return;
}
debug('try to process command');
tmp = processCommand(code);
if (tmp instanceof Error) {
debug( 'getting error %s', tmp.message );
return;
kgryte marked this conversation as resolved.
Show resolved Hide resolved
}
debug('try to compile command');
executable = compileCommand(tmp);
if (executable instanceof Error) {
debug( 'getting error %s', executable.message );
return;
}
try {
if ( this._repl._sandbox ) {
res = executable.compiled.runInContext( this._repl._context, opts );
} else {
res = executable.compiled.runInThisContext( opts );
}
} catch (err) {
debug( 'getting error when executing the command %s', err.message );
return;
}

res = inspect(res);
nlInd = res.indexOf('\n');
if ( nlInd !== -1 ) {
res = res.slice(0, nlInd ) + '...';
}
cursorPosition = this._rli.cursor;
pre = replace( this._repl._outputPrompt, '%d', (this._repl._count+1).toString() );
this._repl._ostream.write( '\n\u001b[90m' + pre + res + '\u001b[0m' );
readline.moveCursor(this._repl._ostream, 0, -1);
readline.cursorTo(this._repl._ostream, cursorPosition + this._repl.promptLength() );

Check warning on line 136 in lib/node_modules/@stdlib/repl/lib/eager_evaluation.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

This line has a length of 88. Maximum allowed is 80
this._isPreview = true;
executable.raw = code;
this._repl._isEagerEvaluated = true;
this._repl._eagerEvaluatedExecutable = executable;
debug( 'sucess' );
});

/**
* Callback which should be invoked**before** a "keypress" event.
*
* @name beforeKeyPress
* @memberof EagerEvaluator.prototype
* @type {Function}
* @param {string} data - input data
* @param {(Object|void)} key - key object
* @returns {void}
*
*/
setNonEnumerableReadOnly(EagerEvaluator.prototype, 'beforeKeypress', function beforeKeyPress(data, key) {
kgryte marked this conversation as resolved.
Show resolved Hide resolved
var cursorPosition;

if (!this._isPreview || key.name === 'return' || (key.name === 'o' && key.ctrl) ) {
return;
}
if ( this._isPreview ) {
cursorPosition = this._rli.cursor;
readline.moveCursor( this._repl._ostream, 0, 1 );
readline.clearLine( this._repl._ostream, 0 );
readline.moveCursor( this._repl._ostream, 0, -1 );
readline.cursorTo( this._repl._ostream, cursorPosition + this._repl.promptLength() );

Check warning on line 166 in lib/node_modules/@stdlib/repl/lib/eager_evaluation.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

This line has a length of 93. Maximum allowed is 80
this._isPreview = false;
}
this._repl._isEagerEvaluated = false;
this._repl._eagerEvaluatedExecutable = void 0;
});

/**
* Tells weather code is side effect free or not.
kgryte marked this conversation as resolved.
Show resolved Hide resolved
*
* @name isSideEffectFree
* @memberof EagerEvaluator.prototype
* @type {Function}
* @param {string} code - input code
* @returns {boolean} - boolean indicate weather code is side effect free or not.
*
*/
setNonEnumerableReadOnly( EagerEvaluator.prototype, 'isSideEffectFree', function isSideEffectFree( code ) {
var ast;
var i;

try {
ast = parse( code, AOPTS );
}
catch (err) {
debug( 'getting error when generating AST %S ', err );
return false;
}
// Iterate from each node in body
for ( i = 0; i < ast.body.length; i++ ) {
if ( !traverse( ast.body[ i ] ) ) {
return false;
}
}
return true;

/**
* Function which recursivly traverse from the node and tells weather node is side effect free or not.
*
* @private
* @param {JSON} node - ast node.
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
* @returns {Boolean} - Boolean indicate weather node is side effect free or not.
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
*/
function traverse(node) {
var fname;
var i;
if ( !node ) {
return false;
}
if ( node.type === 'Literal' || node.type === 'Identifier' || node.type === 'MemberExpression' ) {
return true;
}
if ( node.type === 'BinaryExpression' ) {
if ( traverse( node.left ) && traverse( node.right ) ) {
return true;
}
}
else if ( node.type === 'ExpressionStatement' ) {
if ( traverse( node.expression ) ) {
return true;
}
}
else if ( node.type === 'CallExpression' ) {
fname = getFunctionName( node.callee );
if ( tempDB[fname] && tempDB[fname].isPure ) {
// Iterating through arguments
for ( i = 0; i < node.arguments.length; i++ ) {
if ( !traverse( node.arguments[ i ] ) ) {
return false;
}
}
return true;
}
}
return false;
}

/**
* Get the underscore seprate function name for the member function call.
*
* @private
* @param {JSON} node - ast node
* @returns {string} - underscore seprated function name for the member function call
*/
function getFunctionName( node ) {
if ( !node ) {
return '';
}
if ( node.type === 'MemberExpression' ) {
return getFunctionName(node.object) + '_' + node.property.name;
}
if ( node.type === 'Identifier' ) {
return node.name;
}
return '';
}
} );


// EXPORTS //

module.exports = EagerEvaluator;
23 changes: 22 additions & 1 deletion lib/node_modules/@stdlib/repl/lib/main.js
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@
var PreviewCompleter = require( './completer_preview.js' );
var AutoCloser = require( './auto_close_pairs.js' );
var SyntaxHighlighter = require( './syntax_highlighter.js' );
var EagerEvaluator = require( './eager_evaluation.js' );
var ALIAS_OVERRIDES = require( './alias_overrides.js' );
var SETTINGS = require( './settings.js' );
var SETTINGS_VALIDATORS = require( './settings_validators.js' );
Expand Down Expand Up @@ -130,7 +131,7 @@
* // Close the REPL:
* repl.close();
*/
function REPL( options ) {

Check warning on line 134 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

Function 'REPL' has too many statements (105). Maximum allowed is 100
var ostream;
var themes;
var opts;
Expand Down Expand Up @@ -160,6 +161,14 @@
opts.settings.autoDisableBracketedPasteOnExit = ( opts.settings.autoDisableBracketedPasteOnExit === void 0 ) ? opts.isTTY : opts.settings.autoDisableBracketedPasteOnExit; // eslint-disable-line max-len
opts.settings.syntaxHighlighting = ( opts.settings.syntaxHighlighting === void 0 ) ? opts.isTTY : opts.settings.syntaxHighlighting; // eslint-disable-line max-len

// This is temparary because eager evaluation conflicting with the some tests.
if ( options && options.settings && options.settings.eagerEvaluation !== void 0 ) {

Check warning on line 165 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

This line has a length of 87. Maximum allowed is 80
opts.settings.eagerEvaluation = options.settings.eagerEvaluation;
}
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
else {
opts.settings.eagerEvaluation = ( opts.settings.eagerEvaluation === void 0 ) ? opts.isTTY : opts.settings.eagerEvaluation; // eslint-disable-line max-len
}

debug( 'Options: %s', JSON.stringify({
'input': '<readable_stream>',
'output': '<writable_stream>',
Expand Down Expand Up @@ -254,6 +263,12 @@
// Initialize an internal flag indicating whether we've received a `SIGINT` signal:
setNonEnumerable( this, '_SIGINT', false );

// Initialize an internal flag indicating whether command is eagerlyEvaluated:
setNonEnumerable( this, '_isEagerEvaluated', false );

// Initialize as internal variable for caching the compiled object from eagerEvaluation
setNonEnumerable( this, '_eagerEvaluatedExecutable', void 0 );

Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
// Initialize an internal variable for caching the result of the last successfully evaluated command:
setNonEnumerable( this, '_ans', void 0 );

Expand Down Expand Up @@ -293,11 +308,14 @@
// Initialize a syntax-highlighter:
setNonEnumerableReadOnly( this, '_syntaxHighlighter', new SyntaxHighlighter( this, this._ostream, this._settings.syntaxHighlighting ) );

// Initialize a eagerEvaluator:
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
setNonEnumerableReadOnly( this, '_eagerEvaluator', new EagerEvaluator( this, this._rli, this._settings.eagerEvaluation ) );

// Cache a reference to the private readline interface `ttyWrite` to allow calling the method when wanting default behavior:
setNonEnumerableReadOnly( this, '_ttyWrite', this._rli._ttyWrite );

// Overwrite the private `ttyWrite` method to allow processing input before a "keypress" event is triggered:
this._rli._ttyWrite = beforeKeypress; // WARNING: overwriting a private property

Check warning on line 318 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

Unexpected 'warning' comment: 'WARNING: overwriting a private property'

// Add event listeners:
this._rli.on( 'close', onClose );
Expand All @@ -317,9 +335,9 @@
// Write a welcome message:
this._wstream.write( opts.welcome );

// TODO: check whether to synchronously initialize a REPL history file

Check warning on line 338 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

Unexpected 'todo' comment: 'TODO: check whether to synchronously...'

// TODO: check whether to synchronously initialize a REPL log file

Check warning on line 340 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

Unexpected 'todo' comment: 'TODO: check whether to synchronously...'

// Add any provided user-defined themes...
if ( opts.themes ) {
Expand Down Expand Up @@ -363,6 +381,8 @@
completed = self._previewCompleter.beforeKeypress( data, key );
FLG = self._editorActions.beforeKeypress( data, key );

Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
self._eagerEvaluator.beforeKeypress( data, key );

// If ENTER keypress is encountered or if a preview was completed while navigating, gracefully close the completer...
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
Vinit-Pandit marked this conversation as resolved.
Show resolved Hide resolved
if ( completed || ( key && key.name === 'return' ) ) {
self._completerEngine.closeCompleter();
Expand Down Expand Up @@ -398,6 +418,7 @@
self._previewCompleter.clear();
}
self._completerEngine.onKeypress( data, key );
self._eagerEvaluator.onKeypress( data, key );
self._multilineHandler.onKeypress( data, key );
self._syntaxHighlighter.onKeypress();
self._previewCompleter.onKeypress( data, key );
Expand Down Expand Up @@ -509,9 +530,9 @@
// Update the internal command history buffer: [..., <id>, <cmd>, <success>, ...]
self._history.push( self._count, cmd, success );

// TODO: if successful and if necessary, (asynchronously?) write the command to a history file (question: do we only want to write successful commands to the history file? maybe we need to option for limiting to successful commands?)

Check warning on line 533 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

Unexpected 'todo' comment: 'TODO: if successful and if necessary,...'

// TODO: if necessary, (asynchronously?) write the command and result to a log file (JSON serialization?)

Check warning on line 535 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

Unexpected 'todo' comment: 'TODO: if necessary, (asynchronously?)...'
}
}

Expand Down Expand Up @@ -750,7 +771,7 @@

// Before creating a new execution context in a non-sandboxed environment, remove current workspace variables in order to allow garbage collection and avoid memory leaks (e.g., variables/functions declared during a REPL session which might remain bound to the environment `global` after clearing a REPL):
if ( this._sandbox === false ) {
// WARNING: in a non-sandboxed environment, if a global variable is externally introduced during a REPL session (i.e., introduced via a mechanism outside of the REPL environment), we will delete that global variable, which means the following logic may introduce unintended side-effects for this particular edge case (e.g., application code may expect the presence of the subsequently deleted global variable). While not ideal, (a) user applications should not be introducing globals to begin with and (b) the probability of a user running a REPL session, a user clearing that REPL session, AND a global variable being introduced between starting a REPL and clearing the REPL should be negligible.

Check warning on line 774 in lib/node_modules/@stdlib/repl/lib/main.js

View workflow job for this annotation

GitHub Actions / Lint Changed Files

Unexpected 'warning' comment: 'WARNING: in a non-sandboxed environment,...'
tmp = this._context.vars();
for ( i = 0; i < tmp.length; i++ ) {
if ( isConfigurableProperty( this._context, tmp[ i ] ) ) {
Expand Down Expand Up @@ -1449,7 +1470,7 @@
}
nargs = arguments.length;
if ( nargs === 0 ) {
return assign( {}, this._settings );
return assign({}, this._settings );
}
name = arguments[ 0 ];
if ( !isString( name ) ) {
Expand Down
8 changes: 8 additions & 0 deletions lib/node_modules/@stdlib/repl/lib/multiline_handler.js
Original file line number Diff line number Diff line change
Expand Up @@ -621,6 +621,14 @@ setNonEnumerableReadOnly( MultilineHandler.prototype, 'processLine', function pr
return;
}
this._multiline.active = false; // false until proven otherwise
if ( this._repl._isEagerEvaluated ) {
this.resetInput();
this._queue.push( this._repl._eagerEvaluatedExecutable );
drain( this._repl );
this._repl_isEagerEvaluated = false;
this._repl._eagerEvaluatedExecutable = void 0;
return;
}
cmd = this._cmd.join( '\n' );
if ( RE_WHITESPACE.test( cmd ) ) {
this.resetInput();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,9 @@ function defaultSettings() {
'autoDeletePairs': false,
'autoPage': false,
'completionPreviews': false,
'syntaxHighlighting': false
'syntaxHighlighting': false,
'eagerEvaluation': false

};
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ function defaultSettings() {
'autoDisableBracketedPasteOnExit': false,
'completionPreviews': false,
'syntaxHighlighting': false,
'eagerEvaluation': false,
'autoPage': false
};
}
Expand Down
Loading
Loading