JavaScript compatibility #48
Comments
|
asciidoctor-bibtex and bibtex-ruby are pure ruby and I can try to make asciidoctor-bibtex conform to a certain standard so that it can be compiled to JavaScript. asciidoctor-bibtex is such a simple package that the porting shall not be hard on its own. But you would have to check for all its dependencies. |
|
Would this allow to use asciidoc-bibtex in the live preview of vscode? that would be great! |
|
@Nadano Yes, the live preview of VSCode is based on Asciidoctor.js so once this library is transpiled to JavaScript it can be integrated in JavaScript-based tools. |
|
I am not familiar with opal at the moment. How can this be done? |
|
Basically, you will need to add a Rake task to produce a JavaScript file from this Ruby library. require 'opal'
builder = Opal::Builder.new(compiler_options: {
dynamic_require_severity: :error,
})
builder.append_paths 'lib'
builder.build 'asciidoctor-bibtex'
File.write 'asciidoctor-bibtex.js', builder.to_sThat's the easy part! If you open a pull request with this Rake task, I can provide guidance to resolve these issues one step at a time (you will also need to run this task on Travis so we have a shared environment to reproduce issues). |
|
Is there some progress on this topic? I would help, in fact I even tried it a little bit myself, but I neither know ruby nor javascript very well making this quite difficult. @Mogztter For example how does this work with dependencies. For instance asciidoctor.js, I tried specifying it like this: It then stuck here: I then just disabled the require for asciidoctor completly as I saw that somewhere else Similarly I have problems with other libs. I just did again something like Currently it is stuck with |
Yes, you could say that.
This is the right way, Asciidoctor is a runtime dependency, you should not need it to transpile to JavaScript.
No, Having said that, I didn't find any usage of |
How do I know that it's only a runtime issue?
You're right, I did search a little bit with git blame, the code using it was removed already. I forked the project for now and removed it. Next I run into an issue with dynamic require, but I think I solved this: The next issue is that it requires "java".
I don't quite understand it as I thought RUBY_PLATTFORM is "opal". Anyway I just completly excluded compatibility.rb the same way I did with asciidoctor for now (in my local gem). Which is probably a very bad idea. Do you know how to fix this properly The next issue then is this line: Based on your previous answer this seems again to be part of the stdlib and again not to be supported by opal: opal/opal#1749? |
You can simplify the code for Opal by removing the
That's correct but I don't think we are using |
|
Do I then need to fork all these things and modify the source code? Or is there a better / easier way? (At least for the latex gem for instance ... I don't want to fork everything :-)) And just modifying it locally is not soo great.
At least Edit: I see the issue. Opal does try to compile the whole file no matter what the variables are I think. |
You're right, both are defined: https://github.com/opal/opal/blob/master/opal/corelib/constants.rb#L1-L2
It only works when you explicitly check if the engine/platform is (or isn't)
I think the best approach right now is probably to modify the source code locally. Once you get something working, we will need to apply these changes before transpiling probably using monkey patch. If |
|
Okay I got something compiled yay :) What I have done up to now:
So I now try to use the extension:
But instantly get the error message from the js "ERROR: TypeError: $$ is not a function". This looks like jquery if I'm not mistaken. |
Yay! That's a first step, well done 👍🏻
I think you can submit this patch upstream since it's not used anymore.
Could you please copy/paste the complete stacktrace? You should use Opal 0.11.0 to transpile this library (because Opal 1.1.1 and Opal 0.11.0 are not compatible and Asciidoctor.js 2.2.x is using Opal 0.11.0). |
|
Thanks! Using Opal 0.11.0 already helped with that. Next issue with full stack trace: in the bibtext module section of the lib the following line is: Seems to be Removed it and next error. Is this how you deal with these issues? :D The next error I could resolve myself. Now I get this: The module is |
Not really 😆
Ruby regular expression and JavaScript regular expression share some similarities but do not use the same engine and do not support the same features/syntaxes. This regular expression is probably valid in Ruby but invalid in JavaScript. Having said that running
Where do you get this error? Are you using Node? Which version? |
Ah I see
It's from latex/decode/diacritics, the code snippet above or what do you mean? The exact code should be: new RegExp("" + "\\\\(" + ($send(self.macros.$keys(), 'map', [], ($$5 = function (k) {
var self = $$5.$$s || this;
if (k == null) {
k = nil;
};
return $$($nesting, 'Regexp').$escape(k);
}, $$5.$$s = self, $$5.$$arity = 1, $$5)).$join("|")) + ")\\{(\\p{L}\\p{M}*)([[:alpha:]]*)\\}", 'u')The regex should be more than what was outputed. At least the two "()" are missing and possibly the rest in the middle I'm not sure I'm using Node? I guess :-) My Version is: v12.22.3 |
Sorry I wasn't clear.
If you are using Could you please share the extension transpiled to JavaScript so I can troubleshoot/debug on my machine? |
|
I can reproduce this issue, @patterns = [
ruby_19 {
/\\(#{ @macros.keys.map { |k| Regexp.escape(k) }.join('|') })\{(\p{L}\p{M}*)(\p{Alpha}*)\}/ou
},
/\\(o|O|l|L|aa|AA|ae|AE)\b/,
].freeze(I removed |
Correct
Yeah I do, it's v12.22.3
Why does it run in the browser developer console though? Is this Node specific? Her the js Has this something to do with |
I didn't understand 🤔
Thanks!
That's bad 😞 I'm not entirely sure why this is not done at compile time (in |
Perhaps because of: See beginning of file. As far as I can see it's name_parser.rb and parser.rb, which are definitly needed :( Any suggestions how to move forward? This sounds like it might be simpler to rewrite this whole thing manually in javascript 😞 |
|
I'm not familiar with racc but apparently it's possible to regenerate files using the lib/bibtex/name_parser.rb Racc_debug_parser = false
##### State transition tables end #####
# reduce 0 omitted
def _reduce_1(val, _values, result)
result = []
result
end
# reduce 2 omitted
def _reduce_3(val, _values, result)
result = [val[0]]
result
end
def _reduce_4(val, _values, result)
result << val[1]
result
end
def _reduce_5(val, _values, result)
result = Name.new(val[1])
result
endI think you should suggest this change at: https://github.com/inukshuk/bibtex-ruby |
|
Wow awesome! 👍 I regenerated the files in my fork: https://github.com/Leon0402/bibtex-ruby/blob/master/lib/bibtex/name_parser.rb The error message hasn't changed though. Here is a snippet from the js code. And a couple more of "module_eval" all related to these two files. There is only one "eval" left https://github.com/Leon0402/bibtex-ruby/blob/master/lib/bibtex/filters.rb#L10. But this doesn't seem at all related? |
|
There's still a
That's weird... I can't find any
I don't think it's related, here we are using |
|
I figured it out. First there was a problem with another lib we have as an indirect dependency https://github.com/berkmancenter/namae. This seems pretty much the same code generated with racc, so I fixed it with the -l flag. And then it didn't use my newest master of the bibtex fork. And I still don't understand why. I put branch "master" in my gem file and it worked all the time. I now pinned it to the newest ref on master and now it's installing that version with "bundle install". No module_eval anymore :-) The next error is regex again:
These come from names.y in ruby-bibtex. I suppose it will be similar to the fix you had above with [[:xyz:] becoming to \p{xyz}. I'm not sure though how to replace |
|
I'm note 100% sure, but I think I figured this out: It now says I think the /u in the end might be the problem? at least without there is no problem. Can I just leave it away in ruby? Which might come from https://github.com/inukshuk/bibtex-ruby/blob/a2b74add5c5597cc193e307478193d1b566a9237/lib/bibtex/compatibility.rb. But I don't see the problem. |
|
I think the reason is that the unicode property escapes
If you remove the See: https://stackoverflow.com/questions/7344871/javascript-regular-expression-to-catch-kanji
Iconv is a wrapper class for the UNIX 95 iconv() function family. I doubt it will work in a JavaScript environment. module BibTeX
@iconv_replacements = Hash['ä', 'ae', 'ö', 'oe', 'ü', 'ue', 'Ä', 'Ae', 'Ö', 'Oe', 'Ü', 'Ue', 'ß', 'ss']
# Returns +str+ transliterated containing only ASCII characters.
def self.transliterate(str)
str.gsub(/[äöüÄÖÜß]/, @iconv_replacements)
end
end |
I have no idea either, but I replaced it and it at least compiles. We will see if it works in the end :-)
Oh right, yeah! That's an easy fix, thanks. This seems to be https://github.com/inukshuk/bibtex-ruby/blob/master/lib/bibtex/entry/citeproc_converter.rb. Is something with Hash not supported? |
|
Your last error does not ring a bell. Apparently, the star symbol is a "Variable Length Argument List, Asterisk Operator": https://stackoverflow.com/questions/4170037/what-does-the-star-mean-in-ruby 😆 This syntax is probably not supported by Opal. {:date=>:issued}I'm pretty sure we can use another syntax to get the same result. |
|
The following does the same: CSL_FILTER = %w[
date issued
isbn ISBN
booktitle container-title
journal container-title
journaltitle container-title
series collection-title
address publisher-place
pages page
number issue
url URL
doi DOI
pmid PMID
pmcid PMCID
year issued
type genre
school publisher
institution publisher
organization publisher
howpublished publisher
type genre
urldate accessed
]
.map(&:intern)
.each_slice(2)
.to_a.reduce({}) { |memo, element| memo[element[0]] = element[1]; memo}Maybe it compiles? |
|
The following is also working: CSL_FILTER = Hash[%w[
date issued
isbn ISBN
booktitle container-title
journal container-title
journaltitle container-title
series collection-title
address publisher-place
pages page
number issue
url URL
doi DOI
pmid PMID
pmcid PMCID
year issued
type genre
school publisher
institution publisher
organization publisher
howpublished publisher
type genre
urldate accessed
].map(&:intern).each_slice(2).to_a] |
|
hmm, I think I was mistaken, I'm sorry. Based on some more looking in the source code and experiment it seems to have something to do with: Although I'm not sure what line exactly it is or if it's even that file. Based on the error message and the code around that error message, it might be that: This seems to also somewhat match the error message, although I'm not yet sure how the connection with variable.rb is. I will have another look later. Here is the current js file, if you wanna have a look too |
|
You can try to remove the following lines: I did some tests and |
|
Thanks, that works! Our next problem is from csl-ruby. https://github.com/inukshuk/csl-ruby/blob/master/lib/csl/parser.rb#L11 and https://github.com/inukshuk/csl-ruby/blob/master/lib/csl/schema.rb. For using nokogiri, I probably need to add this to the dependencies. But that leads to errors with nokogiri. So I guess we need to take the nokogiri approach and compile that to javascript as well? |
There is no possible way you are going to be able to compile nokogiri to JavaScript. It uses native libraries (libxml2) and that's outside (like way outside) of what Opal can transpile. |
|
I wonder if it would be possible to create an abstraction on top of the |
|
Do you think it's worth it? |
Hard to say, I don't know how much logic we will need to reimplement. It seems that this extension does quite a few things. Having said that, I don't know if The good thing is that evaluating |
Could you outline a little bit more how that replacement works in detail? We have a module CiteProc with some interface using the citeproc and csl libraries. Then we add require 'CiteProc' unless RUBY_PLATTFORM = 'Opal', so it's just ignored when transpiling. So the javascript libraries will have calls to the interface, but missing the interface itself. Does Opal has some mechanism to bundle our own native javascript code? Or do we need to manually put them together? |
That's correct.
Yes, I think we need to do something like: if RUBY_ENGINE == 'opal'
require 'asciidoctor-bibtex/js'
else
require 'asciidoctor-bibtex/citeproc'
endThen, we can provide a JavaScript implementation: asciidoctor-bibtex/js.rb require 'asciidoctor-bibtex/js/citeproc'asciidoctor-bibtex/js/citeproc.rb module AsciidoctorBibtex
`const __cite__ = require('citation-js')`
module CiteProc
def format(citation, style)
%x{
const data = new __cite__(citation)
return data.format('bibliography', {
format: 'html',
template: style
})
}
end
end
end
Everything wrapped in value = `Math.floor(5.95)`In the above example, |
I think it does! I will give it a shot :) |
|
@Mogztter I would like to give this another shot. So I redid the whole process and this time documented it a little bit better :-) I'm again at the step that I have an exported javascript file. But when I try to build a pdf with:
the extension doesn't get used. I figured there might be a problem with registering the extension and had a look at revealjs. So I added such a file https://github.com/asciidoctor/asciidoctor-reveal.js/blob/master/src/asciidoctor-revealjs.tmpl.js#L12 and the corresponding code in rake. Thanks! |
|
Okay I think I actually was not on the right track with the template. I tried it without it and using asciidoctorjs as suggested by @mojavelinux in zulip. This allows me to actually use the extension for now. I got a bunch of errors regarding some Opal methods that are not found like Isn't it possible to use newer versions of Opal? There were quite some releases since then that might make our life easier |
|
You can't use newer versions of Opal than what Asciidoctor.js is tested with. In a sense, you need to think of Opal as being bundled with Asciidoctor.js. They have a very tight relationship and any changes to Opal could break Asciidoctor.js. |
|
Taking the discussion to a zulip topic: https://asciidoctor.zulipchat.com/#narrow/stream/279645-users.2Fasciidoctor.2Ejs/topic/Port.20Asciidoctor.20BibTex/near/279793429 |
Hi,
I'm working on the Asciidoctor Browser extension (which uses Asciidoctor.js) and I got the following question:
As far as I understand, "asciidoctor-bibtex" uses https://github.com/inukshuk/bibtex-ruby so it may not be an easy task to compile/port this library to JavaScript.
Anyway if you think it's possible to compile this library to JavaScript using Opal I can get you set-up 😉
Thanks!
The text was updated successfully, but these errors were encountered: