diff --git a/README.md b/README.md index 6103ea5c..eebddac8 100644 --- a/README.md +++ b/README.md @@ -1,14 +1,8 @@ -SUPER IMPORTANT UPDATE UNTIL THIS UPDATE DISAPPEARS -=================================================== -None of the following applies if you installed the pool AFTER June 2nd 2017, as the installers will do this work for you. - -The pool currently uses a version of LMDB that is not supported in Ubuntu 16.04 at this time. Please run: bash deployment/install_lmdb_tools.sh once from the root of the installation to load the LMDB tools, this will put them somewhere handy on your path, and drop a new alias to them so they can be used to introspect your database. - -If you had installed the pool prior to 6/2/2017, PLEASE make sure you run a npm install before you restart services. - Pool Design/Theory ================== -The nodejs-pool is built around a small series of core daemons that share access to a single LMDB table for tracking of shares, with MySQL being used to centralize configurations and ensure simple access from local/remote nodes. The core daemons follow: +The nodejs-pool is built around a small series of core daemons that share access to a single LMDB table for tracking of shares, +with MySQL being used to centralize configurations and ensure simple access from local/remote nodes. The core daemons follow: + ```text api - Main API for the frontend to use and pull data from. Expects to be hosted at / remoteShare - Main API for consuming shares from remote/local pools. Expects to be hosted at /leafApi @@ -18,98 +12,75 @@ payments - Handles all payments to workers. blockManager - Unlocks blocks and distributes payments into MySQL worker - Does regular processing of statistics and sends status e-mails for non-active miners. ``` -API listens on port 8001, remoteShare listens on 8000 -Xmrpool.net (The refrence implementation) uses the following setup: -* https://xmrpool.net is hosted on it's own server, as the main website is a static frontend -* https://api.xmrpool.net hosts api, remoteShare, longRunner, payments, blockManager, worker, as these must all be hosted with access to the same LMDB database. +API listens on port 8001, remoteShare listens on 8000. -Sample Caddyfile for API: -```text -https://api.xmrpool.net { - proxy /leafApi 127.0.0.1:8000 - proxy / 127.0.0.1:8001 - cors - gzip -} -``` - -It is critically important that your webserver does not truncate the `/leafApi` portion of the URL for the remoteShare daemon, or it will not function! Local pool servers DO use the remoteShare daemon, as this provides a buffer in case of an error with LMDB or another bug within the system, allowing shares and blocks to queue for submission as soon as the leafApi/remoteShare daemons are back up and responding with 200's. +moneroocean.stream (The reference implementation) uses the following setup: +* https://moneroocean.stream is hosted on its own server, as the main website is a static frontend +* https://api.moneroocean.stream hosts api, remoteShare, longRunner, payments, blockManager, worker, as these must all be hosted with access to the same LMDB database. Setup Instructions ================== Server Requirements ------------------- -* 4 Gb Ram -* 2 CPU Cores (with AES_NI) -* 60 Gb SSD-Backed Storage - If you're doing a multi-server install, the leaf nodes do not need this much storage. They just need enough storage to hold the blockchain for your node. The pool comes configured to use up to 24Gb of storage for LMDB. Assuming you have the longRunner worker running, it should never get near this size, but be aware that it /can/ bloat readily if things error, so be ready for this! +* Ubuntu 24.04 (confirmed working) +* 8 Gb Ram +* 2 CPU Cores +* 150 Gb SSD-Backed Storage - If you're doing a multi-server install, the leaf nodes do not need this much storage. They just need enough storage to hold the blockchain for your node. The pool comes configured to use up to 60Gb of storage for LMDB. Assuming you have the longRunner worker running, it should never get near this size, but be aware that it /can/ bloat readily if things error, so be ready for this! * Notably, this happens to be approximately the size of a 4Gb linode instance, which is where the majority of automated deployment testing happened! Pre-Deploy ---------- -* If you're planning on using e-mail, you'll want to setup an account at https://mailgun.com (It's free for 10k e-mails/month!), so you can notify miners. This also serves as the backend for password reset emails, along with other sorts of e-mails from the pool, including pool startup, pool Monerod daemon lags, etc so it's highly suggested! -* Pre-Generate the wallets, or don't, it's up to you! You'll need the addresses after the install is complete, so I'd suggest making sure you have them available. Information on suggested setups are found below. -* If you're going to be offering PPS, PLEASE make sure you load the pool wallet with XMR before you get too far along. Your pool will trigger PPS payments on it's own, and fairly readily, so you need some float in there! -* Make a non-root user, and run the installer from there! +* If you're planning on using e-mail, you'll want to setup an account at https://mailgun.com (It's free for 10k e-mails/month!), so you can notify miners. This also serves as the backend for password reset emails, along with other sorts of e-mails from the pool, including pool startup, pool Monerod daemon lags, etc so it's highly suggested! +* Pre-Generate the wallets, or don't, it's up to you! You'll need the addresses after the install is complete, so I'd suggest making sure you have them available. Information on suggested setups are found below. +* Run installer from root user and it will make new "user" for pool usage later. Deployment via Installer ------------------------ -1. Add your user to `/etc/sudoers`, this must be done so the script can sudo up and do it's job. We suggest passwordless sudo. Suggested line: ` ALL=(ALL) NOPASSWD:ALL`. Our sample builds use: `pooldaemon ALL=(ALL) NOPASSWD:ALL` -2. Run the [deploy script](https://raw.githubusercontent.com/Snipa22/nodejs-pool/master/deployment/deploy.bash) as a **NON-ROOT USER**. This is very important! This script will install the pool to whatever user it's running under! Also. Go get a coffee, this sucker bootstraps the monero installation. -3. Once it's complete, change as `config.json` appropriate. It is pre-loaded for a local install of everything, running on 127.0.0.1. This will work perfectly fine if you're using a single node setup. You will also want to run: source ~/.bashrc This will activate NVM and get things working for the following pm2 steps. -4. You'll need to change the API end point for the frontend code in the `poolui/build/globals.js` and `poolui/build/global.default.js` -- This will usually be `http(s):///api` unless you tweak caddy! -5. Check `config.json` and change as appropriate. The default database directory `/home//pool_db/` is already been created during startup. If you change the `db_storage_path` just make sure your user has write permissions for new path. Run: `pm2 restart api` to reload the API for usage. You'll also want to set `bind_ip` to the external IP of the pool server, and `hostname` to the resolvable hostname for the pool server. `pool_id` is mostly used for multi-server installations to provide unique identifiers in the backend. -6. Hop into the web interface (Should be at `http:///#/admin`), then login with `Administrator/Password123`, **MAKE SURE TO CHANGE THIS PASSWORD ONCE YOU LOGIN**. *<- This step is currently not active, we're waiting for the frontend to catch up! Head down to the Manual SQL Configuration to take a look at what needs to be done by hand for now*. -7. From the admin panel, you can configure all of your pool's settings for addresses, payment thresholds, etc. -8. Once you're happy with the settings, go ahead and start all the pool daemons, commands follow. +2. Run the [deploy script](https://raw.githubusercontent.com/MoneroOcean/nodejs-pool/master/deployment/deploy.bash) as a **ROOT USER**. This is very important! This script will install the pool to new user "user"! Also. Go get a coffee, this sucker bootstraps the monero installation. +3. Once it's complete, change as `config.json` appropriate. It is pre-loaded for a local install of everything, running on 127.0.0.1. This will work perfectly fine if you're using a single node setup. You'll also want to set `bind_ip` to the external IP of the pool server, and `hostname` to the resolvable hostname for the pool server. `pool_id` is mostly used for multi-server installations to provide unique identifiers in the backend. You will also want to run: source ~/.bashrc This will activate NVM and get things working for the following pm2 steps. +4. You'll need to change the API endpoint for the frontend code in the `moneroocean-gui/script.js` -- This will usually be `http(s)://api.` unless you tweak nginx! +5. The default database directory `/home/user/pool_db/` is already been created during startup. If you change the `db_storage_path` just make sure your user has write permissions for new path. Run: `pm2 restart api` to reload the API for usage. +8. Once you're happy with the settings, go ahead and add pool daemon on main/leaf pool node: ```shell cd ~/nodejs-pool/ -pm2 start init.js --name=blockManager --log-date-format="YYYY-MM-DD HH:mm Z" -- --module=blockManager -pm2 start init.js --name=worker --log-date-format="YYYY-MM-DD HH:mm Z" -- --module=worker -pm2 start init.js --name=payments --log-date-format="YYYY-MM-DD HH:mm Z" -- --module=payments -pm2 start init.js --name=remoteShare --log-date-format="YYYY-MM-DD HH:mm Z" -- --module=remoteShare -pm2 start init.js --name=longRunner --log-date-format="YYYY-MM-DD HH:mm Z" -- --module=longRunner -pm2 start init.js --name=pool --log-date-format="YYYY-MM-DD HH:mm Z" -- --module=pool -pm2 restart api +pm2 start init.js --name=pool --kill-timeout 10000 --log-date-format="YYYY-MM-DD HH:mm:ss:SSS Z" -- --module=pool +pm2 save +pm2 startup ``` -Install Script: +Install Script that assumes you have free Cloudflare DNS setup for your web (moneroocean.stream) and API (api.moneroocean.stream) endpoints, CLoudflare API Token from +from https://dash.cloudflare.com/profile/api-tokens with "Zone.Zone (Read), Zone.DNS (Edit)" permissions and email for certbot updates: + ```bash -curl -L https://raw.githubusercontent.com/Snipa22/nodejs-pool/master/deployment/deploy.bash | bash +curl -L https://raw.githubusercontent.com/MoneroOcean/nodejs-pool/master/deployment/deploy.bash | bash -s -- -s moneroocean.stream api.moneroocean.stream "Cloudflare API Token" support@moneroocean.stream +``` + +Leaf server install script (only for pool module that you need to add/start as shown above after install and updating config.json to connect to your main pool DB server): +```bash +curl -L https://raw.githubusercontent.com/MoneroOcean/nodejs-pool/master/deployment/leaf.bash | bash ``` Assumptions for the installer ----------------------------- -The installer assumes that you will be running a single-node instance and using a clean Ubuntu 16.04 server install. The following system defaults are set: +The installer assumes that you will be running a single-node instance and using a clean Ubuntu 24.04 server install. The following system defaults are set: * MySQL Username: pool * MySQL Password: 98erhfiuehw987fh23d * MySQL Host: 127.0.0.1 -* MySQL root access is only permitted as the root user, the password is in `/root/.my.cnf` +* MySQL root access is only permitted as the root user, the password is in `/root/mysql_pass` * SSL Certificate is generated, self-signed, but is valid for Claymore Miners. -* The server installs and deploys Caddy as it's choice of webserver! - -The following raw binaries **MUST BE AVAILABLE FOR IT TO BOOTSTRAP**: -* sudo - -I've confirmed that the default server 16.04 installation has these requirements. +* The server installs and deploys Caddy as it's choice of web server! -The pool comes pre-configured with values for Monero (XMR), these may need to be changed depending on the exact requirements of your coin. Other coins will likely be added down the road, and most likely will have configuration.sqls provided to overwrite the base configurations for their needs, but can be configured within the frontend as well. - -The pool ALSO applies a series of patches: Fluffy Blocks, Additional Open P2P Connections, 128 Txn Bug Fix. If you don't like these, replace the auto-installed monerod fixes! +The pool comes pre-configured with values for Monero (XMR), these may need to be changed depending on the exact requirements of your coin. +Other coins will likely be added down the road, and most likely will have configuration.sqls provided to overwrite the base configurations for their needs, but can be configured within the frontend as well. Wallet Setup ------------ -The pool is designed to have a dual-wallet design, one which is a fee wallet, one which is the live pool wallet. The fee wallet is the default target for all fees owed to the pool owner. PM2 can also manage your wallet daemon, and that is the suggested run state. - -1. Generate your wallets using `/usr/local/src/monero/build/release/bin/monero-wallet-cli` -2. Make sure to save your regeneration stuff! -3. For the pool wallet, store the password in a file, the suggestion is `~/wallet_pass` -4. Change the mode of the file with chmod to 0400: `chmod 0400 ~/wallet_pass` -5. Start the wallet using PM2: `pm2 start /usr/local/src/monero/build/release/bin/monero-wallet-rpc -- --rpc-bind-port 18082 --password-file ~/wallet_pass --wallet-file --disable-rpc-login --trusted-daemon` -6. If you don't use PM2, then throw the wallet into a screen and have fun. +The pool is designed to have a dual-wallet design, one which is a fee wallet, one which is the live pool wallet. +The fee wallet is the default target for all fees owed to the pool owner. PM2 can also manage your wallet daemon, and that is the suggested run state. Manual Setup ------------ @@ -117,47 +88,27 @@ Pretty similar to the above, you may wish to dig through a few other things for Manual SQL Configuration ------------------------ -Until the full frontend is released, the following SQL information needs to be updated by hand in order to bring your pool online, in module/item format. You can also edit the values in sample_config.sql, then import them into SQL directly via an update. +Until the full frontend is released, the following SQL information needs to be updated by hand in order to bring your pool online, in module/item format. You can also edit the values in sample_config.sql, then import them into SQL directly via an update. ``` -Critical/Must be done: -pool/address -pool/feeAddress -general/shareHost - Nice to have: general/mailgunKey general/mailgunURL general/emailFrom -SQL import command: sudo mysql pool < ~/nodejs-pool/sample_config.sql (Adjust name/path as needed!) +SQL import command: ```mysql -u root --password=$(sudo cat /root/mysql_pass)``` ``` -The shareHost configuration is designed to be pointed at wherever the leafApi endpoint exists. For xmrpool.net, we use https://api.xmrpool.net/leafApi. If you're using the automated setup script, you can use: `http:///leafApi`, as Caddy will proxy it. If you're just using localhost and a local pool serv, http://127.0.0.1:8000/leafApi will do you quite nicely +The shareHost configuration is designed to be pointed at wherever the leafApi endpoint exists. For moneroocean.stream, we use https://api.moneroocean.stream/leafApi. +If you're using the automated setup script, you can use: `http:///leafApi`, as Caddy will proxy it. If you're just using localhost and a local pool server, +http://localhost:8000/leafApi will do you quite nicely. -Additional ports can be added as desired, samples can be found at the end of base.sql. If you're not comfortable with the MySQL command line, I highly suggest MySQL Workbench or a similar piece of software (I use datagrip!). Your root MySQL password can be found in `/root/.my.cnf` - -Final Manual Steps ------------------- -Until the main frontend is done, we suggest running the following SQL line: -``` -DELETE FROM pool.users WHERE username = 'Administrator'; -``` -This will remove the administrator user until there's an easier way to change the password. Alternatively, you can change the password to something not known by the public: -``` -UPDATE pool.users SET email='your new password here' WHERE username='Administrator'; -``` -The email field is used as the default password field until the password is changed, at which point, it's hashed and dumped into the password field instead, and using the email field as a password is disabled. - -You should take a look at the [wiki](https://github.com/Snipa22/nodejs-pool/wiki/Configuration-Details) for specific configuration settings in the system. +Additional ports can be added as desired, samples can be found at the end of base.sql. If you're not comfortable with the MySQL command line, +I highly suggest MySQL Workbench or a similar piece of software (I use datagrip!). Your root MySQL password can be found in `/root/mysql_pass` Pool Update Procedures ====================== If upgrading the pool, please do a git pull to get the latest code within the pool's directory. -Once complete, please `cd` into `sql_sync`, then run `node sql_sync.js` - -This will update your pool with the latest config options with any defaults that the pools may set. - Pool Troubleshooting ==================== @@ -214,57 +165,92 @@ Status of shares Overflow pages: 0 Entries: 4379344 ``` -The important thing to verify here is that the "Number of pages used" value is less than the "Max Pages" value, and that there are "Free pages" under "Freelist Status". If this is the case, them look at the "Reader Table Status" and look for the PID listed. Run: +The important thing to verify here is that the "Number of pages used" value is less than the "Max Pages" value, and that there are "Free pages" under "Freelist Status". +If this is the case, them look at the "Reader Table Status" and look for the PID listed. Run: ```shell ps fuax | grep ex: ps fuax | grep 25763 ``` -If the output is not blank, then one of your node processes is reading, this is fine. If there is no output given on one of them, then proceed forwards. + +If the output is not blank, then one of your node processes is reading, this is fine. If there is no output given on one of them, then proceed forwards. The second step is to run: ```shell -pm2 stop blockManager worker payments remoteShare longRunner api -pm2 start blockManager worker payments remoteShare longRunner api +pm2 restart blockManager worker payments remoteShare longRunner api ``` This will restart all of your related daemons, and will clear any open reader connections, allowing LMDB to get back to a normal state. -If on the other hand, you have no "Free pages" and your Pages used is equal to the Max Pages, then you've run out of disk space for LMDB. You need to verify the cleaner is working. For reference, 4.3 million shares are stored within approximately 2-3 Gb of space, so if you're vastly exceeding this, then your cleaner (longRunner) is likely broken. - - -PPS Fee Thoughts -================ -If you're considering PPS, I've spoken with [Fireice_UK](https://github.com/fireice-uk/) whom kindly did some math about what you're looking at in terms of requiements to run a PPS pool without it self-impoloding under particular risk factors, based on the work found [here](https://arxiv.org/pdf/1112.4980.pdf) - -```text -Also I calculated the amount of XMR needed to for a PPS pool to stay afloat. Perhaps you should put them up in the README to stop some spectacular clusterfucks :D: -For 1 in 1000000 chance that the pool will go bankrupt: 5% fee -> 1200 2% fee -> 3000 -For 1 in 1000000000 chance: 5% fee -> 1800 2% fee -> 4500 -``` - -The developers of the pool have not verified this, but based on our own usage on https://xmrpool.net/ this seems rather reasonable. You should be wary if you're consdering PPS and take you fees into account appropriately! +If on the other hand, you have no "Free pages" and your Pages used is equal to the Max Pages, then you've run out of disk space for LMDB. +You need to verify the cleaner is working. For reference, 4.3 million shares are stored within approximately 2-3 Gb of space, +so if you're vastly exceeding this, then your cleaner (longRunner) is likely broken. Installation/Configuration Assistance ===================================== -If you need help installing the pool from scratch, please have your servers ready, which would be Ubuntu 16.04 servers, blank and clean, DNS records pointed. These need to be x86_64 boxes with AES-NI Available. +If you need help installing the pool from scratch, please have your servers ready, which would be Ubuntu 16.04 servers, blank and clean, DNS records pointed. +These need to be x86_64 boxes with AES-NI Available. -Installation asstiance is 7 XMR, with a 3 XMR deposit, with remainder to be paid on completion. -Configuration assistance is 4 XMR with a 2 XMR deposit, and includes debugging your pool configurations, ensuring that everything is running, and tuning for your uses/needs. +Installation assistance is 3 XMR, with a 1 XMR deposit, with remainder to be paid on completion. +Configuration assistance is 2 XMR with a 1 XMR deposit, and includes debugging your pool configurations, ensuring that everything is running, and tuning for your uses/needs. +altblockManager module source (that determines the most profitable coin to mine and trades them to main coin on exchanges, in particular Tradeogre, Xeggex, Binance, Coinex, Gate, Livecoin, Okex, Sevenseas) +price is 20 XMR. SSH access with a sudo-enabled user will be needed, preferably the user that is slated to run the pool. -If you'd like assistance with setting up node-cryptonote-pool, please provide what branch/repo you'd like to work from, as there's a variety of these. - Assistance is not available for frontend customization at this time. -For assitance, please contact Snipa at pool_install@snipanet.com or via IRC at irc.freenode.net in the #monero-pools channel. +For assistance, please contact MoneroOcean at support@moneroocean.stream. + +Developer Donations +=================== +If you'd like to make a one time donation, the addresses are as follows: +* XMR - ```89TxfrUmqJJcb1V124WsUzA78Xa3UYHt7Bg8RGMhXVeZYPN8cE5CZEk58Y1m23ZMLHN7wYeJ9da5n5MXharEjrm41hSnWHL``` +* AEON - ```WmsEg3RuUKCcEvFBtXcqRnGYfiqGJLP1FGBYiNMgrcdUjZ8iMcUn2tdcz59T89inWr9Vae4APBNf7Bg2DReFP5jr23SQqaDMT``` +* ETN - ```etnkQMp3Hmsay2p7uxokuHRKANrMDNASwQjDUgFb5L2sDM3jqUkYQPKBkooQFHVWBzEaZVzfzrXoETX6RbMEvg4R4csxfRHLo1``` +* SUMO - ```Sumoo1DGS7c9LEKZNipsiDEqRzaUB3ws7YHfUiiZpx9SQDhdYGEEbZjRET26ewuYEWAZ8uKrz6vpUZkEVY7mDCZyGnQhkLpxKmy``` +* GRFT - ```GACadqdXj5eNLnyNxvQ56wcmsmVCFLkHQKgtaQXNEE5zjMDJkWcMVju2aYtxbTnZgBboWYmHovuiH1Ahm4g2N5a7LuMQrpT``` +* MSR - ```5hnMXUKArLDRue5tWsNpbmGLsLQibt23MEsV3VGwY6MGStYwfTqHkff4BgvziprTitbcDYYpFXw2rEgXeipsABTtEmcmnCK``` +* LTHN - ```iz53aMEaKJ25zB8xku3FQK5VVvmu2v6DENnbGHRmn659jfrGWBH1beqAzEVYaKhTyMZcxLJAdaCW3Kof1DwTiTbp1DSqLae3e``` +* WOW - ```Wo3yjV8UkwvbJDCB1Jy7vvXv3aaQu3K8YMG6tbY3Jo2KApfyf5RByZiBXy95bzmoR3AvPgNq6rHzm98LoHTkzjiA2dY7sqQMJ``` +* XMV - ```XvyVfpAYp3zSuvdtoHgnDzMUf7GAeiumeUgVC7RTq6SfgtzGEzy4dUgfEEfD5adk1kN4dfVZdT3zZdgSD2xmVBs627Vwt2C3Ey``` +* RYO - ```RYoLsi22qnoKYhnv1DwHBXcGe9QK6P9zmekwQnHdUAak7adFBK4i32wFTszivQ9wEPeugbXr2UD7tMd6ogf1dbHh76G5UszE7k1``` +* XLA - ```SvkpUizij25ZGRHGb1c8ZTAHp3VyNFU3NQuQR1PtMyCqdpoZpaYAGMfG99z5guuoktY13nrhEerqYNKXvoxD7cUM1xA6Z5rRY``` +* XHV - ```hvxyEmtbqs5TEk9U2tCxyfGx2dyGD1g8EBspdr3GivhPchkvnMHtpCR2fGLc5oEY42UGHVBMBANPge5QJ7BDXSMu1Ga2KFspQR``` +* TUBE - ```TubedBNkgkTbd2CBmLQSwW58baJNghD9xdmctiRXjrW3dE8xpUcoXimY4J5UMrnUBrUDmfQrbxRYRX9s5tQe7pWYNF2QiAdH1Fh``` +* LOKI - ```L6XqN6JDedz5Ub8KxpMYRCUoQCuyEA8EegEmeQsdP5FCNuXJavcrxPvLhpqY6emphGTYVrmAUVECsE9drafvY2hXUTJz6rW``` +* TRTL - ```TRTLv2x2bac17cngo1r2wt3CaxN8ckoWHe2TX7dc8zW8Fc9dpmxAvhVX4u4zPjpv9WeALm2koBLF36REVvsLmeufZZ1Yx6uWkYG``` +* XTNC - ```XtazhSxz1bbJLpT2JuiD2UWFUJYSFty5SVWuF6sy2w9v8pn69smkUxkTVCQc8NKCd6CBMNDGzgdPRYBKaHdbgZ5SNptVH1yPCTQ``` +* IRD - ```ir3DHyB8Ub1aAHEewMeUxQ7b7tQdWa7VL8M5oXDPohS3Me4nhwvALXM4mym2kWg9VsceT75dm6XWiWF1K4zu8RVQ1HJD8Z3R9``` +* ARQ - ```ar4Ha6ZQCkKRhkKQLfexv7VZQM2MhUmMmU9hmzswCPK4T3o2rbPKZM1GxEoYg4AFQsh57PsEets7sbpU958FAvxo2RkkTQ1gE``` +* XWP - ```fh4MCJrakhWGoS6Meqp6UxGE1GNfAjKaRdPjW36rTffDiqvEq2HWEKZhrbYRw7XJb3CXxkjL3tcYGTT39m5qgjvk1ap4bVu1R``` +* XEQ - ```Tvzp9tTmdGP9X8hCEw1Qzn18divQajJYTjR5HuUzHPKyLK5fzRt2X73FKBDzcnHMDJKdgsPhUDVrKHVcDJQVmLBg33NbkdjQb``` +* XTA - ```ipN5cNhm7RXAGACP4ZXki4afT3iJ1A6Ka5U4cswE6fBPDcv8JpivurBj3vu1bXwPyb8KZEGsFUYMmToFG4N9V9G72X4WpAQ8L``` +* DERO - ```dero1qygrgnz9gea2rqgwhdtpfpa3mvagt5uyq0g92nurwrpk6wnn7hdnzqgudsv6t``` +* CCX - ```ccx7dmnBBoRPuVcpKJSAVZKdSDo9rc7HVijFbhG34jsXL3qiqfRwu7A5ecem44s2rngDd8y8N4QnYK6WR3mXAcAZ5iXun9BQBx``` +* BLOC - ```abLoc5iUG4a6oAb2dqygxkS5M2uHWx16zHb9fUWMzpSEDwm6T7PSq2MLdHonWZ16CGfnJKRomq75aZyviTo6ZjHeYQMzNAEkjMg``` +* ZEPH - ```ZEPHYR2nic7ULkkmgZNX8a9i2tMbkxuCqjgWZYuee3awX7RhtmhoT98CwGEGrruWZVSKtA7Z7JC8m7oeYHtBD9cBEZzdEh9BSdq4q``` +* SAL - ```SaLvdWKnkz6MvVgxXr2TWSDSvESz6EBcz3wmMFch2sQuMYz2sUQGVNDYhkYaSuvkDr9GSYp5h6BeQHnGK8HzKhqGeZCZzG3AHS3``` +* RVN - ```RLVJv9rQNHzXS3Zn4JH8hfAHmm1LfECMxy``` +* XNA - ```Nb931jkFtFN7QWpu4FqSThaoKajYjS5iFZ``` +* CLORE - ```AdXPHtV8yb86a8QKsbs8gmUpRpcxufRn8n``` +* RTM - ```RUCyaEZxQu3Eure73XPQ57si813RYAMQKC``` +* KCN - ```kc1qchtxq2gw9dc4r58hcegd6n4jspew6j9mu3yz8q``` +* BTRM - ```Bfhtr2g56tg73TNZBRCu6fJUD39Kur6SGG``` +* ERG - ```9fe533kUzAE57YfPP6o3nzsYMKN2W2uCxvg8KG8Vn5DDeJGetRw``` +* BTC - ```3HRbMgcvbqHVW7P34MNGvF2Gh3DE26iHdw``` +* BCH - ```18sKoDSjLCFW9kZrXuza1qzEERnKi7bx8S``` +* ETH - ```0xfE23a61548FCCE159a541FAe9e16cEB92Da650ed``` +* ETC - ```0x4480Ad73a113BEFf05B2079E38D90c9757Ecb063``` +* LTC - ```MGj8PU1PpTNDDqRHmuEqfDpH3gxp6cJrUU``` Credits ======= [Zone117x](https://github.com/zone117x) - Original [node-cryptonote-pool](https://github.com/zone117x/node-cryptonote-pool) from which, the stratum implementation has been borrowed. +[Snipa](https://github.com/Snipa22) - Original [nodejs-pool](https://github.com/Snipa22/nodejs-pool) from which, the original implementation has been borrowed. + [Mesh00](https://github.com/mesh0000) - Frontend build in Angular JS [XMRPoolUI](https://github.com/mesh0000/poolui) [Wolf0](https://github.com/wolf9466/)/[OhGodAGirl](https://github.com/ohgodagirl) - Rebuild of node-multi-hashing with AES-NI [node-multi-hashing](https://github.com/Snipa22/node-multi-hashing-aesni) diff --git a/SQL_MIGRATIONS.md b/SQL_MIGRATIONS.md deleted file mode 100644 index 355b13d8..00000000 --- a/SQL_MIGRATIONS.md +++ /dev/null @@ -1,35 +0,0 @@ -2/12/2017 ---------- -```sql -ALTER TABLE pool.config MODIFY item_value TEXT; -``` - -2/13/2017 ---------- -```sql -ALTER TABLE pool.payments ADD transfer_fee BIGINT(20) DEFAULT 0 NULL; -``` - -2/16/2017 ---------- -```sql -ALTER DATABASE pool DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.balance CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.bans CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.block_log CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.config CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.payments CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.pools CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.port_config CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.ports CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.shapeshiftTxn CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.transactions CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.users CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -ALTER TABLE pool.xmrtoTxn CONVERT TO CHARACTER SET utf8 COLLATE utf8_general_ci; -``` - -2/25/2017 ---------- -```sql -ALTER TABLE pool.users ADD enable_email BOOL DEFAULT true NULL; -``` \ No newline at end of file diff --git a/block_notify.sh b/block_notify.sh new file mode 100755 index 00000000..dce9fe99 --- /dev/null +++ b/block_notify.sh @@ -0,0 +1,2 @@ +#!/bin/bash +/bin/echo 18081 | /bin/nc -N localhost 2223 \ No newline at end of file diff --git a/block_share_dumps/calc_mo_cvs.js b/block_share_dumps/calc_mo_cvs.js new file mode 100644 index 00000000..7d67e6b5 --- /dev/null +++ b/block_share_dumps/calc_mo_cvs.js @@ -0,0 +1,79 @@ +"use strict"; + +if (Boolean(process.stdin.isTTY) || process.argv.length !== 3) { + console.log("Usage: unxz -c .cvs.xz | node calc_mo_cvs.js "); + console.log(" wget -O - https://block-share-dumps.moneroocean.stream/.cvs.xz | unxz -c | node calc_mo_cvs.js "); + process.exit(1); +} + +const my_wallet = process.argv[2].slice(-16); + +let stdin = ""; + +process.stdin.on('data', function(data) { + stdin += data.toString(); +}); + +function human_hashrate(hashes) { + const power = Math.pow(10, 2 || 0); + if (hashes > 1000000000000) return String(Math.round((hashes / 1000000000000) * power) / power) + " TH/s"; + if (hashes > 1000000000) return String(Math.round((hashes / 1000000000) * power) / power) + " GH/s"; + if (hashes > 1000000) return String(Math.round((hashes / 1000000) * power) / power) + " MH/s"; + if (hashes > 1000) return String(Math.round((hashes / 1000) * power) / power) + " KH/s"; + return Math.floor( hashes || 0 ) + " H/s" +}; + +process.stdin.on('end', function() { + let pplns_window = 0; + let oldest_timestamp = 0; + let newest_timestamp = 0; + + let my_share_count = 0; + let my_xmr_diff = 0; + let my_xmr_diff_payed = 0; + let my_coin_raw_diff = {}; + let my_coin_xmr_diff = {}; + + for (let line of stdin.split("\n")) { + if (line.substring(0, 1) == "#") continue; + const items = line.split('\t'); + if (items.length < 7) { + console.error("Skipped invalid line: " + line); + continue; + } + const wallet = items[0]; + const timestamp = parseInt(items[1], 16); + const raw_diff = parseInt(items[2]); + const count = parseInt(items[3]); + const coin = items[4]; + const xmr_diff = parseInt(items[5]); + const xmr_diff_payed = items[6] == "" ? xmr_diff : parseInt(items[6]); + pplns_window += xmr_diff; + if (!oldest_timestamp || timestamp < oldest_timestamp) oldest_timestamp = timestamp; + if (newest_timestamp < timestamp) newest_timestamp = timestamp; + if (wallet === my_wallet) { + my_share_count += count; + my_xmr_diff += xmr_diff; + my_xmr_diff_payed += xmr_diff_payed; + if (!(coin in my_coin_raw_diff)) my_coin_raw_diff[coin] = 0; + my_coin_raw_diff[coin] += raw_diff; + if (!(coin in my_coin_xmr_diff)) my_coin_xmr_diff[coin] = 0; + my_coin_xmr_diff[coin] += xmr_diff; + } + } + + console.log("PPLNS window size: \t" + ((newest_timestamp - oldest_timestamp)/1000/60/60).toFixed(2) + " hours"); + console.log("PPLNS window size: \t" + pplns_window + " xmr hashes"); + console.log("Pool XMR normalized hashrate: \t" + human_hashrate(pplns_window / (newest_timestamp - oldest_timestamp) * 1000)); + console.log(""); + console.log("Your submitted shares: \t" + my_share_count); + console.log("Your payment: \t" + ((my_xmr_diff_payed / pplns_window) * 100).toFixed(6) + "% (" + my_xmr_diff_payed + " xmr hashes)"); + console.log("Your XMR normalized hashrate: \t" + human_hashrate(my_xmr_diff_payed / (newest_timestamp - oldest_timestamp) * 1000)); + console.log(""); + console.log("You mined these coins:"); + for (let coin of Object.keys(my_coin_raw_diff).sort()) { + console.log("\t" + coin + ": " + my_coin_raw_diff[coin] + " raw coin hashes (" + ((my_coin_xmr_diff[coin] / my_xmr_diff) * 100).toFixed(6) + "% of XMR normalized hashrate)"); + } + + process.exit(0); +}); \ No newline at end of file diff --git a/block_share_dumps/calc_mo_cvs_top.js b/block_share_dumps/calc_mo_cvs_top.js new file mode 100644 index 00000000..2cc85aca --- /dev/null +++ b/block_share_dumps/calc_mo_cvs_top.js @@ -0,0 +1,75 @@ +"use strict"; + +if (Boolean(process.stdin.isTTY) || process.argv.length !== 2) { + console.log("Usage: unxz -c .cvs.xz | node calc_mo_cvs_top.js"); + console.log(" wget -O - https://block-share-dumps.moneroocean.stream/.cvs.xz | unxz -c | node calc_mo_cvs_top.js"); + process.exit(1); +} + +let stdin = ""; + +process.stdin.on('data', function(data) { + stdin += data.toString(); +}); + +function human_hashrate(hashes) { + const power = Math.pow(10, 2 || 0); + if (hashes > 1000000000000) return String(Math.round((hashes / 1000000000000) * power) / power) + " TH/s"; + if (hashes > 1000000000) return String(Math.round((hashes / 1000000000) * power) / power) + " GH/s"; + if (hashes > 1000000) return String(Math.round((hashes / 1000000) * power) / power) + " MH/s"; + if (hashes > 1000) return String(Math.round((hashes / 1000) * power) / power) + " KH/s"; + return Math.floor( hashes || 0 ) + " H/s" +}; + +process.stdin.on('end', function() { + let pplns_window = 0; + let oldest_timestamp = 0; + let newest_timestamp = 0; + + let wallets = {}; + + let my_share_count = 0; + let my_xmr_diff = 0; + let my_xmr_diff_payed = 0; + let my_coin_raw_diff = {}; + let my_coin_xmr_diff = {}; + + for (let line of stdin.split("\n")) { + if (line.substring(0, 1) == "#") continue; + const items = line.split('\t'); + if (items.length < 7) { + console.error("Skipped invalid line: " + line); + continue; + } + const wallet = items[0]; + const timestamp = parseInt(items[1], 16); + const raw_diff = parseInt(items[2]); + const count = parseInt(items[3]); + const coin = items[4]; + const xmr_diff = parseInt(items[5]); + const xmr_diff_payed = items[6] == "" ? xmr_diff : parseInt(items[6]); + pplns_window += xmr_diff; + if (!oldest_timestamp || timestamp < oldest_timestamp) oldest_timestamp = timestamp; + if (newest_timestamp < timestamp) newest_timestamp = timestamp; + if (!(wallet in wallets)) wallets[wallet] = { + share_count: 0, + xmr_diff: 0, + xmr_diff_payed: 0, + coin_raw_diff: {}, + coin_xmr_diff: {}, + }; + wallets[wallet].share_count += count; + wallets[wallet].xmr_diff += xmr_diff; + wallets[wallet].xmr_diff_payed += xmr_diff_payed; + if (!(coin in wallets[wallet].coin_raw_diff)) wallets[wallet].coin_raw_diff[coin] = 0; + wallets[wallet].coin_raw_diff[coin] += raw_diff; + if (!(coin in wallets[wallet].coin_xmr_diff)) wallets[wallet].coin_xmr_diff[coin] = 0; + wallets[wallet].coin_xmr_diff[coin] += xmr_diff; + } + + for (let wallet of Object.keys(wallets).sort((a, b) => (wallets[a].xmr_diff < wallets[b].xmr_diff) ? 1 : -1)) { + console.log(wallet + ": " + wallets[wallet].xmr_diff); + } + + process.exit(0); +}); \ No newline at end of file diff --git a/coinConfig.json b/coinConfig.json index e1595a75..cdf6e355 100644 --- a/coinConfig.json +++ b/coinConfig.json @@ -8,21 +8,5 @@ "name": "Monero", "mixIn": 4, "shortCode": "XMR" - }, - "krb": { - "funcFile": "./lib/coins/krb.js", - "paymentFile": "./payment_systems/krb.js", - "sigDigits": 1000000000000, - "name": "Karbowanec", - "mixIn": 4, - "shortCode": "KRB" - }, - "aeon": { - "funcFile": "./lib/coins/aeon.js", - "paymentFile": "./payment_systems/aeon.js", - "sigDigits": 1000000000000, - "name": "Aeon Coin", - "mixIn": 4, - "shortCode": "AEON" } } diff --git a/config_example.json b/config_example.json index cf3ea8a5..78d1944c 100644 --- a/config_example.json +++ b/config_example.json @@ -1,8 +1,11 @@ { "pool_id": 0, + "eth_pool_support": 1, + "worker_num": 0, "bind_ip": "127.0.0.1", "hostname": "testpool.com", "db_storage_path": "CHANGEME", + "verify_shares_host": null, "coin": "xmr", "mysql": { "connectionLimit": 20, diff --git a/debug_scripts/block_add.js b/debug_scripts/block_add.js deleted file mode 100644 index d1fa2a70..00000000 --- a/debug_scripts/block_add.js +++ /dev/null @@ -1,91 +0,0 @@ -"use strict"; -let mysql = require("promise-mysql"); -let fs = require("fs"); -let argv = require('minimist')(process.argv.slice(2)); -let config = fs.readFileSync("../config.json"); -let coinConfig = fs.readFileSync("../coinConfig.json"); -let protobuf = require('protocol-buffers'); -const request = require('request'); - -global.support = require("../lib/support.js")(); -global.config = JSON.parse(config); -global.mysql = mysql.createPool(global.config.mysql); -global.protos = protobuf(fs.readFileSync('../lib/data.proto')); -let comms; -let coinInc; - - -// Config Table Layout -// . - -global.mysql.query("SELECT * FROM config").then(function (rows) { - rows.forEach(function (row){ - if (!global.config.hasOwnProperty(row.module)){ - global.config[row.module] = {}; - } - if (global.config[row.module].hasOwnProperty(row.item)){ - return; - } - switch(row.item_type){ - case 'int': - global.config[row.module][row.item] = parseInt(row.item_value); - break; - case 'bool': - global.config[row.module][row.item] = (row.item_value === "true"); - break; - case 'string': - global.config[row.module][row.item] = row.item_value; - break; - case 'float': - global.config[row.module][row.item] = parseFloat(row.item_value); - break; - } - }); -}).then(function(){ - global.config['coin'] = JSON.parse(coinConfig)[global.config.coin]; - coinInc = require("." + global.config.coin.funcFile); - global.coinFuncs = new coinInc(); - if (argv.module === 'pool'){ - comms = require('../lib/remote_comms'); - } else { - comms = require('../lib/local_comms'); - } - global.database = new comms(); - global.database.initEnv(); - global.coinFuncs.blockedAddresses.push(global.config.pool.address); - global.coinFuncs.blockedAddresses.push(global.config.payout.feeAddress); -}).then(function(){ - /* - message Block { - required string hash = 1; - required int64 difficulty = 2; - required int64 shares = 3; - required int64 timestamp = 4; - required POOLTYPE poolType = 5; - required bool unlocked = 6; - required bool valid = 7; - optional int64 value = 8; - } - */ - let invalidBlockProto = global.protos.Block.encode({ - hash: "88cf2c37e1e4e8a273cbe3ec502b6975fd6c4ebe1e8889ad9d5e53a5e9cde007", - difficulty: 1002932, - shares: 0, - timestamp: Date.now(), - poolType: global.protos.POOLTYPE.PPS, - unlocked: false, - valid: true, - value:0 - }); - let wsData = global.protos.WSData.encode({ - msgType: global.protos.MESSAGETYPE.BLOCK, - key: global.config.api.authKey, - msg: invalidBlockProto, - exInt: 1 - }); - request.post({url: global.config.general.shareHost, body: wsData}, function (error, response, body) { - console.log(error); - console.log(JSON.stringify(response)); - console.log(JSON.stringify(body)); - }); -}); \ No newline at end of file diff --git a/debug_scripts/block_locker.js b/debug_scripts/block_locker.js deleted file mode 100644 index e164eb0e..00000000 --- a/debug_scripts/block_locker.js +++ /dev/null @@ -1,19 +0,0 @@ -"use strict"; -let mysql = require("promise-mysql"); -let fs = require("fs"); -let argv = require('minimist')(process.argv.slice(2)); -let config = fs.readFileSync("../config.json"); -let coinConfig = fs.readFileSync("../coinConfig.json"); -let protobuf = require('protocol-buffers'); - -global.support = require("../lib/support.js")(); -global.config = JSON.parse(config); -global.mysql = mysql.createPool(global.config.mysql); -global.protos = protobuf(fs.readFileSync('../lib/data.proto')); -let comms; -comms = require('../lib/local_comms'); -global.database = new comms(); -global.database.initEnv(); -global.database.lockBlock(argv.blockID); -console.log("Block "+argv.blockID+" re-locked! Exiting!"); -process.exit(); \ No newline at end of file diff --git a/debug_scripts/block_migrator_from_old_sql.js b/debug_scripts/block_migrator_from_old_sql.js deleted file mode 100644 index b23db7c4..00000000 --- a/debug_scripts/block_migrator_from_old_sql.js +++ /dev/null @@ -1,105 +0,0 @@ -"use strict"; -let mysql = require("promise-mysql"); -let fs = require("fs"); -let argv = require('minimist')(process.argv.slice(2)); -let config = fs.readFileSync("../config.json"); -let coinConfig = fs.readFileSync("../coinConfig.json"); -let protobuf = require('protocol-buffers'); -const request = require('request'); - -global.support = require("../lib/support.js")(); -global.config = JSON.parse(config); -global.mysql = mysql.createPool(global.config.mysql); -global.protos = protobuf(fs.readFileSync('../lib/data.proto')); -let comms; -let coinInc; - - -// Config Table Layout -// . - -global.mysql.query("SELECT * FROM config").then(function (rows) { - rows.forEach(function (row){ - if (!global.config.hasOwnProperty(row.module)){ - global.config[row.module] = {}; - } - if (global.config[row.module].hasOwnProperty(row.item)){ - return; - } - switch(row.item_type){ - case 'int': - global.config[row.module][row.item] = parseInt(row.item_value); - break; - case 'bool': - global.config[row.module][row.item] = (row.item_value === "true"); - break; - case 'string': - global.config[row.module][row.item] = row.item_value; - break; - case 'float': - global.config[row.module][row.item] = parseFloat(row.item_value); - break; - } - }); -}).then(function(){ - global.config['coin'] = JSON.parse(coinConfig)[global.config.coin]; - coinInc = require("." + global.config.coin.funcFile); - global.coinFuncs = new coinInc(); - if (argv.module === 'pool'){ - comms = require('../lib/remote_comms'); - } else { - comms = require('../lib/local_comms'); - } - global.database = new comms(); - global.database.initEnv(); - global.coinFuncs.blockedAddresses.push(global.config.pool.address); - global.coinFuncs.blockedAddresses.push(global.config.payout.feeAddress); -}).then(function(){ - /* - message Block { - required string hash = 1; - required int64 difficulty = 2; - required int64 shares = 3; - required int64 timestamp = 4; - required POOLTYPE poolType = 5; - required bool unlocked = 6; - required bool valid = 7; - optional int64 value = 8; - } - */ - global.mysql.query("SELECT * FROM blocks").then(function(rows){ - rows.forEach(function(row){ - let block = { - hash: row.hex, - difficulty: row.difficulty, - shares: row.shares, - timestamp: global.support.formatDateFromSQL(row.find_time)*1000, - poolType: null, - unlocked: row.unlocked === 1, - valid: row.valid === 1 - }; - switch(row.pool_type){ - case 'pplns': - block.poolType = global.protos.POOLTYPE.PPLNS; - break; - case 'solo': - block.poolType = global.protos.POOLTYPE.SOLO; - break; - case 'prop': - block.poolType = global.protos.POOLTYPE.PROP; - break; - case 'pps': - block.poolType = global.protos.POOLTYPE.PPS; - break; - default: - block.poolType = global.protos.POOLTYPE.PPLNS; - } - global.coinFuncs.getBlockHeaderByHash(block.hash, function(header){ - block.value = header.reward; - let txn = global.database.env.beginTxn(); - txn.putBinary(global.database.blockDB, row.height, global.protos.Block.encode(block)); - txn.commit(); - }); - }); - }); -}); \ No newline at end of file diff --git a/debug_scripts/block_share_fix.js b/debug_scripts/block_share_fix.js deleted file mode 100644 index f2f3d715..00000000 --- a/debug_scripts/block_share_fix.js +++ /dev/null @@ -1,59 +0,0 @@ -"use strict"; -let mysql = require("promise-mysql"); -let fs = require("fs"); -let argv = require('minimist')(process.argv.slice(2)); -let config = fs.readFileSync("../config.json"); -let coinConfig = fs.readFileSync("../coinConfig.json"); -let protobuf = require('protocol-buffers'); -const request = require('request'); - -global.support = require("../lib/support.js")(); -global.config = JSON.parse(config); -global.mysql = mysql.createPool(global.config.mysql); -global.protos = protobuf(fs.readFileSync('../lib/data.proto')); -let comms; -let coinInc; - - -// Config Table Layout -// . - -global.mysql.query("SELECT * FROM config").then(function (rows) { - rows.forEach(function (row){ - if (!global.config.hasOwnProperty(row.module)){ - global.config[row.module] = {}; - } - if (global.config[row.module].hasOwnProperty(row.item)){ - return; - } - switch(row.item_type){ - case 'int': - global.config[row.module][row.item] = parseInt(row.item_value); - break; - case 'bool': - global.config[row.module][row.item] = (row.item_value === "true"); - break; - case 'string': - global.config[row.module][row.item] = row.item_value; - break; - case 'float': - global.config[row.module][row.item] = parseFloat(row.item_value); - break; - } - }); -}).then(function(){ - global.config['coin'] = JSON.parse(coinConfig)[global.config.coin]; - coinInc = require("." + global.config.coin.funcFile); - global.coinFuncs = new coinInc(); - if (argv.module === 'pool'){ - comms = require('../lib/remote_comms'); - } else { - comms = require('../lib/local_comms'); - } - global.database = new comms(); - global.database.initEnv(); - global.coinFuncs.blockedAddresses.push(global.config.pool.address); - global.coinFuncs.blockedAddresses.push(global.config.payout.feeAddress); -}).then(function(){ - global.database.fixBlockShares(1241110); -}); \ No newline at end of file diff --git a/debug_scripts/socket_io.html b/debug_scripts/socket_io.html deleted file mode 100644 index bce2a717..00000000 --- a/debug_scripts/socket_io.html +++ /dev/null @@ -1,22 +0,0 @@ - - - - \ No newline at end of file diff --git a/deployment/base.sql b/deployment/base.sql index 9bb15288..3d754c0e 100644 --- a/deployment/base.sql +++ b/deployment/base.sql @@ -1,12 +1,16 @@ CREATE DATABASE pool; -GRANT ALL ON pool.* TO pool@`127.0.0.1` IDENTIFIED BY '98erhfiuehw987fh23d'; -GRANT ALL ON pool.* TO pool@localhost IDENTIFIED BY '98erhfiuehw987fh23d'; +CREATE USER pool@`127.0.0.1` IDENTIFIED WITH mysql_native_password BY '98erhfiuehw987fh23d'; +CREATE USER pool@`172.17.0.1` IDENTIFIED WITH mysql_native_password BY '98erhfiuehw987fh23d'; +CREATE USER pool@localhost IDENTIFIED WITH mysql_native_password BY '98erhfiuehw987fh23d'; +GRANT ALL ON pool.* TO pool@`127.0.0.1`; +GRANT ALL ON pool.* TO pool@`172.17.0.1`; +GRANT ALL ON pool.* TO pool@localhost; FLUSH PRIVILEGES; USE pool; ALTER DATABASE pool DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci; CREATE TABLE `balance` ( `id` int(11) NOT NULL AUTO_INCREMENT, - `last_edited` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + `last_edited` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, `payment_address` varchar(128) DEFAULT NULL, `payment_id` varchar(128) DEFAULT NULL, `pool_type` varchar(64) DEFAULT NULL, @@ -17,26 +21,45 @@ CREATE TABLE `balance` ( UNIQUE KEY `balance_payment_address_pool_type_bitcoin_payment_id_uindex` (`payment_address`,`pool_type`,`bitcoin`,`payment_id`), KEY `balance_payment_address_payment_id_index` (`payment_address`,`payment_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; +CREATE TABLE `paid_blocks` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `paid_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + `found_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, + `port` int NOT NULL, + `hex` varchar(128) NOT NULL, + `amount` bigint(20) DEFAULT NULL, + PRIMARY KEY (`id`), + KEY `paid_blocks_paid_time` (`paid_time`), + UNIQUE KEY `paid_blocks_hex` (`hex`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; +CREATE TABLE `block_balance` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `hex` varchar(128) NOT NULL, + `payment_address` varchar(128) DEFAULT NULL, + `payment_id` varchar(128) DEFAULT NULL, + `amount` float(53) DEFAULT NULL, + PRIMARY KEY (`id`), + UNIQUE KEY `block_balance_id_uindex` (`id`), + UNIQUE KEY `block_balance_hex_payment_address_payment_id_uindex` (`hex`, `payment_address`,`payment_id`), + KEY `block_balance_hex_index` (`hex`), + KEY `block_balance_payment_address_payment_id_index` (`payment_address`,`payment_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `bans` ( `id` int(11) NOT NULL AUTO_INCREMENT, `ip_address` varchar(40) DEFAULT NULL, `mining_address` varchar(200) DEFAULT NULL, + `reason` varchar(200) DEFAULT NULL, `active` tinyint(1) DEFAULT '1', `ins_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP, PRIMARY KEY (`id`), UNIQUE KEY `bans_id_uindex` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; -CREATE TABLE `block_log` ( - `id` int(11) NOT NULL COMMENT 'Block Height', - `orphan` tinyint(1) DEFAULT '1', - `hex` varchar(128) NOT NULL, - `find_time` timestamp NULL DEFAULT NULL, - `reward` bigint(20) DEFAULT NULL, - `difficulty` bigint(20) DEFAULT NULL, - `major_version` int(11) DEFAULT NULL, - `minor_version` int(11) DEFAULT NULL, - PRIMARY KEY (`hex`), - UNIQUE KEY `block_log_hex_uindex` (`hex`) +CREATE TABLE `notifications` ( + `id` int(11) NOT NULL AUTO_INCREMENT, + `mining_address` varchar(200) DEFAULT NULL, + `message` varchar(200) DEFAULT NULL, + PRIMARY KEY (`id`), + UNIQUE KEY `notifications_id_uindex` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `config` ( `id` int(11) NOT NULL AUTO_INCREMENT, @@ -74,9 +97,17 @@ CREATE TABLE `pools` ( `blockID` int(11) DEFAULT NULL, `blockIDTime` timestamp NULL DEFAULT NULL, `hostname` varchar(128) DEFAULT NULL, + `port` int DEFAULT NULL, PRIMARY KEY (`id`), UNIQUE KEY `pools_id_uindex` (`id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; +CREATE TABLE `pool_workers` ( + `id` tinyint(1) unsigned NOT NULL AUTO_INCREMENT, + `pool_id` int(11) NOT NULL, + `worker_id` int(11) NOT NULL, + PRIMARY KEY (`id`), + UNIQUE KEY `pool_workers_id_uindex` (`pool_id`, `worker_id`) +) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `port_config` ( `poolPort` int(11) NOT NULL, `difficulty` int(11) DEFAULT '1000', @@ -137,13 +168,14 @@ CREATE TABLE `transactions` ( KEY `transactions_shapeshiftTxn_id_fk` (`exchange_txn_id`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8; CREATE TABLE `users` ( - `id` int(11) NOT NULL AUTO_INCREMENT, + `id` int(11) unsigned NOT NULL AUTO_INCREMENT, `username` varchar(256) NOT NULL, `pass` varchar(64) DEFAULT NULL, `email` varchar(256) DEFAULT NULL, `admin` tinyint(1) DEFAULT '0', `payout_threshold` bigint(16) DEFAULT '0', `enable_email` tinyint(1) DEFAULT '1', + `payout_threshold_lock` tinyint(1) DEFAULT '0', PRIMARY KEY (`id`), UNIQUE KEY `users_id_uindex` (`id`), UNIQUE KEY `users_username_uindex` (`username`) @@ -173,17 +205,68 @@ INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'banPercent', '25', 'int', 'Percentage of shares that need to be invalid to be banned.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'banThreshold', '30', 'int', 'Number of shares before bans can begin'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustedMiners', 'true', 'bool', 'Enable the miner trust system'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minerThrottleSharePerSec', '5', 'int', 'Number of shares per second (per thread) after pool will throttle shares'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minerThrottleShareWindow', '5', 'int', 'Length of share throttle window in seconds'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'workerMax', '1000', 'int', 'Max number of worker connection before pool starts to issue bans'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'shareAccTime', '60', 'int', 'Length of time shares are accumulated in seconds'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustChange', '1', 'int', 'Change in the miner trust in percent'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustMin', '20', 'int', 'Minimum level of miner trust'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'trustPenalty', '30', 'int', 'Number of shares that must be successful to be trusted, reset to this value if trust share is broken'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'retargetTime', '60', 'int', 'Time between difficulty retargets'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'address', '127.0.0.1', 'string', 'Monero Daemon RPC IP'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'port', '18081', 'int', 'Monero Daemon RPC Port'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address', '127.0.0.1', 'string', 'Monero Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'basicAuth', '', 'string', 'Basic auth header if needed by daemon'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'X-API-KEY', '', 'string', 'Turtle wallet API auth header'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'pollInterval', '100', 'int', 'Time in ms between pool daemon checks for new blocks'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'pollBlockInterval', '5000', 'int', 'Time in ms between pool daemon checks for block template updates'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'maxBlockKeepTime', '30', 'int', 'Mix block template update time in seconds on the same height'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorRYO', '0', 'float', 'Ryo algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorSUMO', '0', 'float', 'SUMO algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorLOKI', '0', 'float', 'Loki algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXRN', '0', 'float', 'Saronite algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXTNC', '0', 'float', 'XtendCash algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorWOW', '0', 'float', 'Wownero algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorTUBE', '0', 'float', 'BitTube algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXHV', '0', 'float', 'Haven algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorAEON', '0', 'float', 'Aeon algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorMSR', '0', 'float', 'Masari algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXLA', '0', 'float', 'Scala algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorLTHN', '0', 'float', 'Lethean algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorGRFT', '0', 'float', 'Graft algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorTRTL', '0', 'float', 'Turtle algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorIRD', '0', 'float', 'Iridium algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorARQ', '0', 'float', 'ArqMa algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXMV', '0', 'float', 'MoneroV algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXWP', '0', 'float', 'Swap algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXEQ', '0', 'float', 'Equilibria algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorCCX', '0', 'float', 'Conceal algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXTA', '0', 'float', 'Italocoin algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorDERO', '0', 'float', 'Dero algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXMC', '0', 'float', 'XMC algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorRVN', '0', 'float', 'RVN algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorETH', '0', 'float', 'ETH algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorETC', '0', 'float', 'ETC algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorBLOC', '0', 'float', 'BLOC algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorERG', '0', 'float', 'ERG algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorRTM', '0', 'float', 'RTM algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorKCN', '0', 'float', 'KCN algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorBTRM', '0', 'float', 'BTRM algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorZEPH', '0', 'float', 'ZEPH algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorXNA', '0', 'float', 'XNA algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorCLORE', '0', 'float', 'CLORE algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'coinHashFactorSAL', '0', 'float', 'SAL algo hash price factor relative to coinHashFactor'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'enableAlgoSwitching', 'false', 'bool', 'Enable smart miners (need additional altblockManager module)'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('daemon', 'verifyHost', '', 'string', 'Use to extra daemon height verify check'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address', '127.0.0.1', 'string', 'Default Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_18082', '127.0.0.1', 'string', 'Monero Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_8545', '127.0.0.1', 'string', 'ETH Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_8645', '127.0.0.1', 'string', 'ETC Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_8766', '127.0.0.1', 'string', 'RVN Daemon RPC Wallet IP'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'address_9053', '127.0.0.1', 'string', 'ERG Daemon RPC Wallet IP'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('wallet', 'port', '18082', 'int', 'Monero Daemon RPC Wallet Port'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('rpc', 'https', 'false', 'bool', 'Enable RPC over SSL'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'maxDifficulty', '500000', 'int', 'Maximum difficulty for VarDiff'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minDifficulty', '100', 'int', 'Minimum difficulty for VarDiff'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'maxDifficulty', '10000000000000', 'int', 'Maximum difficulty for VarDiff'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'minDifficulty', '10000', 'int', 'Minimum difficulty for VarDiff'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'varDiffVariance', '20', 'int', 'Percentage out of the target time that difficulty changes'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'varDiffMaxChange', '125', 'int', 'Percentage amount that the difficulty may change'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'btcFee', '1.5', 'float', 'Fee charged for auto withdrawl via BTC'); @@ -191,46 +274,93 @@ INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'pplnsFee', '.6', 'float', 'Fee charged for the usage of the PPLNS pool'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'propFee', '.7', 'float', 'Fee charged for the usage of the proportial pool'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'soloFee', '.4', 'float', 'Fee charged for usage of the solo mining pool'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'exchangeMin', '5', 'float', 'Minimum XMR balance for payout to exchange/payment ID'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'walletMin', '.3', 'float', 'Minimum XMR balance for payout to personal wallet'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'exchangeMin', '.1', 'float', 'Minimum XMR balance for payout to exchange/payment ID'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'walletMin', '.01', 'float', 'Minimum XMR balance for payout to personal wallet'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'defaultPay', '.1', 'float', 'Default XMR balance for payout'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'devDonation', '3', 'float', 'Donation to XMR core development'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'poolDevDonation', '3', 'float', 'Donation to pool developer'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'denom', '.000001', 'float', 'Minimum balance that will be paid out to.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'blocksRequired', '60', 'int', 'Blocks required to validate a payout before it''s performed.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'blocksRequired', '30', 'int', 'Blocks required to validate a payout before it''s performed.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'anchorRound', '1', 'int', 'Round anchor height to group payment block pre-calc better. 1 - no round, 2 - round to every even block, 3 - round to every 3-rd block, etc.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'sigDivisor', '1000000000000', 'int', 'Divisor for turning coin into human readable amounts '); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feesForTXN', '10', 'int', 'Amount of XMR that is left from the fees to pay miner fees.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'maxTxnValue', '250', 'int', 'Maximum amount of XMR to send in a single transaction'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'shapeshiftPair', 'xmr_btc', 'string', 'Pair to use in all shapeshift lookups for auto BTC payout'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'coinCode', 'XMR', 'string', 'Coincode to be loaded up w/ the shapeshift getcoins argument.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'allowBitcoin', 'false', 'bool', 'Allow the pool to auto-payout to BTC via ShapeShift'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'exchangeRate', '0', 'float', 'Current exchange rate'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'bestExchange', 'xmrto', 'string', 'Current best exchange'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'mixIn', '4', 'int', 'Mixin count for coins that support such things.'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'statsBufferLength', '480', 'int', 'Number of items to be cached in the stats buffers.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'mixIn', '10', 'int', 'Mixin count for coins that support such things.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'ethWalletPass', '', 'string', 'Ethereum wallet password'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'statsBufferLength', '1000', 'int', 'Number of items to be cached in the stats buffers.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'statsBufferHours', '72', 'int', 'Number of hours to be cached in the stats buffers.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pps', 'enable', 'false', 'bool', 'Enable PPS or not'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pplns', 'shareMulti', '2', 'int', 'Multiply this times difficulty to set the N in PPLNS'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pplns', 'shareMultiLog', '3', 'int', 'How many times the difficulty of the current block do we keep in shares before clearing them out'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'blockCleaner', 'true', 'bool', 'Enable the deletion of blocks or not.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address', '', 'string', 'Address to mine to, this should be the wallet-rpc address.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19734', '', 'string', 'Address to mine to for 19734 (SUMO) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_26968', '', 'string', 'Address to mine to for 26968 (ETN) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_18981', '', 'string', 'Address to mine to for 18981 (GRFT) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_38081', '', 'string', 'Address to mine to for 38081 (MSR) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_48782', '', 'string', 'Address to mine to for 48782 (ITNS) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_34568', '', 'string', 'Address to mine to for 34568 (WOW) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19281', '', 'string', 'Address to mine to for 19281 (XMV) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_12211', '', 'string', 'Address to mine to for 12211 (RYO) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_11181', '', 'string', 'Address to mine to for 11181 (Aeon) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_17750', '', 'string', 'Address to mine to for 17750 (Haven) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_25182', '', 'string', 'Address to mine to for 25182 (BitTube) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_11812', '', 'string', 'Address to mine to for 11812 (Scala) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_22023', '', 'string', 'Address to mine to for 22023 (Loki) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_31014', '', 'string', 'Address to mine to for 31014 (Saronite) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_33124', '', 'string', 'Address to mine to for 33124 (XtendCash) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_11898', '', 'string', 'Address to mine to for 11898 (Turtle) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_13007', '', 'string', 'Address to mine to for 13007 (Iridium) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19994', '', 'string', 'Address to mine to for 19994 (ArqMa) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19950', '', 'string', 'Address to mine to for 19950 (Swap) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_13102', '', 'string', 'Address to mine to for 13102 (Italocoin) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_20206', '', 'string', 'Address to mine to for 20206 (Dero) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_18181', '', 'string', 'Address to mine to for 18181 (XMC) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_16000', '', 'string', 'Address to mine to for 16000 (CCX) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_9231', '', 'string', 'Address to mine to for 9231 (Equilibria) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_8766', '', 'string', 'Address to mine to for 8766 (Ravencoin) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_8545', '', 'string', 'Address to mine to for 8545 (Ethereum) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_8645', '', 'string', 'Address to mine to for 8645 (Ethereum Classic) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_2086', '', 'string', 'Address to mine to for 2086 (BLOC) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_9053', '', 'string', 'Address to mine to for 9053 (ERG) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_9998', '', 'string', 'Address to mine to for 9998 (RTM) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_5110', '', 'string', 'Address to mine to for 5110 (KCN) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_17767', '', 'string', 'Address to mine to for 17767 (ZEPH) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_10225', '', 'string', 'Address to mine to for 10225 (BTRM) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19001', '', 'string', 'Address to mine to for 19001 (Neurai) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_9766', '', 'string', 'Address to mine to for 9766 (CLORE) port.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pool', 'address_19081', '', 'string', 'Address to mine to for 19081 (SAL) port.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeAddress', '', 'string', 'Address that pool fees are sent to.'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'cmcKey', '', 'string', 'CMC API Key for notification'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'mailgunKey', '', 'string', 'MailGun API Key for notification'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'mailgunURL', '', 'string', 'MailGun URL for notifications'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'mailgunNoCert', 'false', 'bool', 'Disable certificate check for MailGun'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'emailFrom', '', 'string', 'From address for the notification emails'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'testnet', 'false', 'bool', 'Does this pool use testnet?'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'blockCleanWarning', '360', 'int', 'Blocks before longRunner cleaner module will start to warn.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('pplns', 'enable', 'true', 'bool', 'Enable PPLNS on the pool.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('solo', 'enable', 'true', 'bool', 'Enable SOLO mining on the pool'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeSlewAmount', '.011', 'float', 'Amount to charge for the txn fee'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeSlewAmount', '.0001', 'float', 'Amount to charge for the txn fee'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'feeSlewEnd', '4', 'float', 'Value at which txn fee amount drops to 0'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'rpcPasswordEnabled', 'false', 'bool', 'Does the wallet use a RPC password?'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'rpcPasswordPath', '', 'string', 'Path and file for the RPC password file location'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'maxPaymentTxns', '5', 'int', 'Maximum number of transactions in a single payment'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'shareHost', '', 'string', 'Host that receives share information'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerNotHashingBody', 'Hello,\n\nYour worker: %(worker)s has stopped submitting hashes at: %(timestamp)s UTC\n\nThank you,\n%(poolEmailSig)s', 'string', 'Email sent to the miner when their worker stops hashing'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerNotHashingSubject', 'Worker %(worker)s stopped hashing', 'string', 'Subject of email sent to miner when worker stops hashing'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'maxPaymentTxns', '15', 'int', 'Maximum number of transactions in a single payment'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'shareHost', 'http://localhost/leafApi', 'string', 'Host that receives share information'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerNotHashingBody', 'Your worker: %(worker)s has stopped submitting hashes at: %(timestamp)s UTC\n', 'string', 'Email sent to the miner when their worker stops hashing'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerNotHashingSubject', 'Status of your worker(s)', 'string', 'Subject of email sent to miner when worker stops hashing'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerStartHashingBody', 'Your worker: %(worker)s has started submitting hashes at: %(timestamp)s UTC\n', 'string', 'Email sent to the miner when their worker starts hashing'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('email', 'workerStartHashingSubject', 'Status of your worker(s)', 'string', 'Subject of email sent to miner when worker starts hashing'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'adminEmail', '', 'string', 'Email of pool admin for alert notification stuff'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'emailSig', 'NodeJS-Pool Administration Team', 'string', 'Signature line for the emails.'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'timer', '120', 'int', 'Number of minutes between main payment daemon cycles'); INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'timerRetry', '25', 'int', 'Number of minutes between payment daemon retrying due to not enough funds'); -INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'priority', '0', 'int', 'Payout priority setting. 0 = use default (4x fee); 1 = low prio (1x fee)'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('payout', 'priority', '1', 'int', 'Payout priority setting. 0 = use default (4x fee); 1 = low prio (1x fee)'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'allowStuckPoolKill', 'false', 'bool', 'Allow to kill the pool in case of stuck block template'); +INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('general', 'dbSizeGB', '24', 'int', 'LMDB size in GBs'); INSERT INTO pool.users (username, pass, email, admin, payout_threshold) VALUES ('Administrator', null, 'Password123', 1, 0); INSERT INTO pool.port_config (poolPort, difficulty, portDesc, portType, hidden, `ssl`) VALUES (3333, 1000, 'Low-End Hardware (Up to 30-40 h/s)', 'pplns', 0, 0); INSERT INTO pool.port_config (poolPort, difficulty, portDesc, portType, hidden, `ssl`) VALUES (5555, 5000, 'Medium-Range Hardware (Up to 160 h/s)', 'pplns', 0, 0); diff --git a/deployment/caddyfile b/deployment/caddyfile deleted file mode 100644 index 9352fcfe..00000000 --- a/deployment/caddyfile +++ /dev/null @@ -1,18 +0,0 @@ -# Catch-all vhost -:80 { - root /var/www - proxy /leafApi 127.0.0.1:8000 - proxy /api 127.0.0.1:8001 { - without /api - } - proxy /socket.io 127.0.0.1:8001 { - header_upstream Connection {>Connection} - header_upstream Upgrade {>Upgrade} - } - header / { - Access-Control-Allow-Methods "GET, POST, OPTIONS" - Access-Control-Allow-Headers "Content-Type, x-access-token" - } - gzip - mime .manifest text/cache-manifest -} diff --git a/deployment/deploy.bash b/deployment/deploy.bash index 96c6ddad..afd6e146 100644 --- a/deployment/deploy.bash +++ b/deployment/deploy.bash @@ -1,98 +1,237 @@ -#!/bin/bash -echo "This assumes that you are doing a green-field install. If you're not, please exit in the next 15 seconds." -sleep 15 -echo "Continuing install, this will prompt you for your password if you're not already running as root and you didn't enable passwordless sudo. Please do not run me as root!" -if [[ `whoami` == "root" ]]; then - echo "You ran me as root! Do not run me as root!" - exit 1 +#!/bin/bash -x + +NODEJS_VERSION=v20.12.2 + +WWW_DNS=$1 +API_DNS=$2 +CF_DNS_API_TOKEN=$3 +CERTBOT_EMAIL=$4 + +test -z $WWW_DNS && WWW_DNS="moneroocean.stream" +test -z $API_DNS && API_DNS="api.moneroocean.stream" +test -z $CF_DNS_API_TOKEN && CF_DNS_API_TOKEN="n/a" +test -z $CERTBOT_EMAIL && CERTBOT_EMAIL="support@moneroocean.stream" + +if [[ $(whoami) != "root" ]]; then + echo "Please run this script as root" + exit 1 fi -ROOT_SQL_PASS=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) -CURUSER=$(whoami) -echo "Etc/UTC" | sudo tee -a /etc/timezone -sudo rm -rf /etc/localtime -sudo ln -s /usr/share/zoneinfo/Zulu /etc/localtime -sudo dpkg-reconfigure -f noninteractive tzdata -sudo apt-get update -sudo DEBIAN_FRONTEND=noninteractive apt-get -y upgrade -sudo debconf-set-selections <<< "mysql-server mysql-server/root_password password $ROOT_SQL_PASS" -sudo debconf-set-selections <<< "mysql-server mysql-server/root_password_again password $ROOT_SQL_PASS" -echo -e "[client]\nuser=root\npassword=$ROOT_SQL_PASS" | sudo tee /root/.my.cnf -sudo DEBIAN_FRONTEND=noninteractive apt-get -y install git python-virtualenv python3-virtualenv curl ntp build-essential screen cmake pkg-config libboost-all-dev libevent-dev libunbound-dev libminiupnpc-dev libunwind8-dev liblzma-dev libldns-dev libexpat1-dev libgtest-dev mysql-server lmdb-utils libzmq3-dev -cd ~ -git clone https://github.com/Snipa22/nodejs-pool.git # Change this depending on how the deployment goes. -cd /usr/src/gtest -sudo cmake . -sudo make -sudo mv libg* /usr/lib/ -cd ~ -sudo systemctl enable ntp + +DEBIAN_FRONTEND=noninteractive apt-get update +DEBIAN_FRONTEND=noninteractive apt-get full-upgrade -y +timedatectl set-timezone Etc/UTC + +adduser --disabled-password --gecos "" user +grep -q "user ALL=(ALL) NOPASSWD:ALL" /etc/sudoers || echo "user ALL=(ALL) NOPASSWD:ALL" >>/etc/sudoers +su user -c "mkdir /home/user/.ssh" +if [ -f "/root/.ssh/authorized_keys" ]; then + mv /root/.ssh/authorized_keys /home/user/.ssh/authorized_keys + chown user:user /home/user/.ssh/authorized_keys + chmod 600 /home/user/.ssh/authorized_keys + sed -i 's/#\?PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config + sed -i 's/#\?PermitRootLogin .\+/PermitRootLogin no/g' /etc/ssh/sshd_config + sed -i 's/#\?PermitEmptyPasswords .\+/PermitEmptyPasswords no/g' /etc/ssh/sshd_config + service ssh restart +fi + +ufw default deny incoming +ufw default allow outgoing +ufw allow ssh +ufw allow 443 +ufw --force enable + +cat >/root/.vimrc <<'EOF' +colorscheme desert +set fo-=ro +EOF + +cat >/home/user/.vimrc <<'EOF' +colorscheme desert +set fo-=ro +EOF +chown user:user /home/user/.vimrc + +DEBIAN_FRONTEND=noninteractive apt-get install -y nginx ntp sudo +snap install --classic certbot +snap set certbot trust-plugin-with-root=ok +snap install certbot-dns-cloudflare +find /snap/certbot -name options-ssl-nginx.conf | xargs -I{} cp {} /etc/letsencrypt/options-ssl-nginx.conf +echo "dns_cloudflare_api_token=$CF_DNS_API_TOKEN" >/root/dns_cloudflare_api_token.ini +chmod 600 /root/dns_cloudflare_api_token.ini +certbot certonly --non-interactive --agree-tos --email "$CERTBOT_EMAIL" --dns-cloudflare --dns-cloudflare-propagation-seconds 30 --dns-cloudflare-credentials /root/dns_cloudflare_api_token.ini -d $WWW_DNS +certbot certonly --non-interactive --agree-tos --email "$CERTBOT_EMAIL" --dns-cloudflare --dns-cloudflare-propagation-seconds 30 --dns-cloudflare-credentials /root/dns_cloudflare_api_token.ini -d $API_DNS +cat >/etc/nginx/sites-enabled/default <~/wallets/wallet_pass +echo 1 | /usr/local/src/monero/build/release/bin/monero-wallet-cli --offline --create-address-file --generate-new-wallet ~/wallets/wallet --password-file ~/wallets/wallet_pass --command address +echo 1 | /usr/local/src/monero/build/release/bin/monero-wallet-cli --offline --create-address-file --generate-new-wallet ~/wallets/wallet_fee --password-file ~/wallets/wallet_pass --command address +EOF +) | su user -l +echo; echo; echo +read -p "*** Write down your seeds for wallet and wallet_fee listed above and press ENTER to continue ***" + +cat >/lib/systemd/system/monero.service <<'EOF' +[Unit] +Description=Monero Daemon +After=network.target + +[Service] +ExecStart=/usr/local/src/monero/build/release/bin/monerod --hide-my-port --prune-blockchain --enable-dns-blocklist --no-zmq --out-peers 64 --non-interactive --restricted-rpc --block-notify '/bin/bash /home/user/nodejs-pool/block_notify.sh' +Restart=always +User=monerodaemon +Nice=10 +CPUQuota=400% + +[Install] +WantedBy=multi-user.target +EOF + +useradd -m monerodaemon -d /home/monerodaemon +systemctl daemon-reload +systemctl enable monero +systemctl start monero + +sleep 30 +echo "Please wait until Monero daemon is fully synced" +tail -f /home/monerodaemon/.bitmonero/bitmonero.log 2>/dev/null | grep Synced & +( tail -F -n0 /home/monerodaemon/.bitmonero/bitmonero.log & ) | egrep -q "You are now synchronized with the network" +killall tail 2>/dev/null +echo "Monero daemon is synced" + +DEBIAN_FRONTEND=noninteractive apt-get install -y mysql-server +ROOT_SQL_PASS=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1) +(cat </root/mysql_pass +chmod 600 /root/mysql_pass +grep max_connections /etc/mysql/my.cnf || cat >>/etc/mysql/my.cnf <<'EOF' +[mysqld] +max_connections = 10000 +EOF +systemctl restart mysql + +(cat <>/home/user/.bashrc + echo 'export PATH=/home/user/.bin:$PATH' >>/home/user/.bashrc + for i in mdb_copy mdb_dump mdb_load mdb_stat; do cp \$i /home/user/.bin/; done +) npm install -g pm2 +pm2 install pm2-logrotate openssl req -subj "/C=IT/ST=Pool/L=Daemon/O=Mining Pool/CN=mining.pool" -newkey rsa:2048 -nodes -keyout cert.key -x509 -out cert.pem -days 36500 -mkdir ~/pool_db/ -sed -r "s/(\"db_storage_path\": ).*/\1\"\/home\/$CURUSER\/pool_db\/\",/" config_example.json > config.json -cd ~ -git clone https://github.com/mesh0000/poolui.git -cd poolui -npm install -./node_modules/bower/bin/bower update -./node_modules/gulp/bin/gulp.js build -cd build -sudo ln -s `pwd` /var/www -CADDY_DOWNLOAD_DIR=$(mktemp -d) -cd $CADDY_DOWNLOAD_DIR -curl -sL "https://snipanet.com/caddy.tar.gz" | tar -xz caddy init/linux-systemd/caddy.service -sudo mv caddy /usr/local/bin -sudo chown root:root /usr/local/bin/caddy -sudo chmod 755 /usr/local/bin/caddy -sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/caddy -sudo groupadd -g 33 www-data -sudo useradd -g www-data --no-user-group --home-dir /var/www --no-create-home --shell /usr/sbin/nologin --system --uid 33 www-data -sudo mkdir /etc/caddy -sudo chown -R root:www-data /etc/caddy -sudo mkdir /etc/ssl/caddy -sudo chown -R www-data:root /etc/ssl/caddy -sudo chmod 0770 /etc/ssl/caddy -sudo cp ~/nodejs-pool/deployment/caddyfile /etc/caddy/Caddyfile -sudo chown www-data:www-data /etc/caddy/Caddyfile -sudo chmod 444 /etc/caddy/Caddyfile -sudo sh -c "sed 's/ProtectHome=true/ProtectHome=false/' init/linux-systemd/caddy.service > /etc/systemd/system/caddy.service" -sudo chown root:root /etc/systemd/system/caddy.service -sudo chmod 644 /etc/systemd/system/caddy.service -sudo systemctl daemon-reload -sudo systemctl enable caddy.service -sudo systemctl start caddy.service -rm -rf $CADDY_DOWNLOAD_DIR -cd ~ -sudo env PATH=$PATH:`pwd`/.nvm/versions/node/v6.9.2/bin `pwd`/.nvm/versions/node/v6.9.2/lib/node_modules/pm2/bin/pm2 startup systemd -u $CURUSER --hp `pwd` -cd ~/nodejs-pool -sudo chown -R $CURUSER. ~/.pm2 -echo "Installing pm2-logrotate in the background!" -pm2 install pm2-logrotate & -mysql -u root --password=$ROOT_SQL_PASS < deployment/base.sql -mysql -u root --password=$ROOT_SQL_PASS pool -e "INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('api', 'authKey', '`cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1`', 'string', 'Auth key sent with all Websocket frames for validation.')" -mysql -u root --password=$ROOT_SQL_PASS pool -e "INSERT INTO pool.config (module, item, item_value, item_type, Item_desc) VALUES ('api', 'secKey', '`cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 32 | head -n 1`', 'string', 'HMAC key for Passwords. JWT Secret Key. Changing this will invalidate all current logins.')" +mkdir /home/user/pool_db +sed -r 's#("db_storage_path": ).*#\1"/home/user/pool_db/",#' config_example.json >config.json +mysql -u root --password=$ROOT_SQL_PASS > ~/.bashrc -echo 'export PATH=~/.bin:$PATH' >> ~/.bashrc -for i in mdb_copy mdb_dump mdb_load mdb_stat; do cp $i ~/.bin/; done -echo "Please run source ~/.bashrc to initialize the new LMDB tools. Thanks for flying Snipa22 Patch Services." \ No newline at end of file diff --git a/deployment/leaf.bash b/deployment/leaf.bash index e7127ca7..aa092883 100644 --- a/deployment/leaf.bash +++ b/deployment/leaf.bash @@ -1,52 +1,99 @@ -#!/bin/bash -echo "This assumes that you are doing a green-field install. If you're not, please exit in the next 15 seconds." -sleep 15 -echo "Continuing install, this will prompt you for your password if you're not already running as root and you didn't enable passwordless sudo. Please do not run me as root!" -if [[ `whoami` == "root" ]]; then - echo "You ran me as root! Do not run me as root!" - exit 1 +#!/bin/bash -x + +NODEJS_VERSION=v20.12.2 + +if [[ $(whoami) != "root" ]]; then + echo "Please run this script as root" + exit 1 fi -CURUSER=$(whoami) -echo "Etc/UTC" | sudo tee -a /etc/timezone -sudo rm -rf /etc/localtime -sudo ln -s /usr/share/zoneinfo/Zulu /etc/localtime -sudo dpkg-reconfigure -f noninteractive tzdata -sudo apt-get update -sudo DEBIAN_FRONTEND=noninteractive apt-get -y upgrade -sudo DEBIAN_FRONTEND=noninteractive apt-get -y install git python-virtualenv python3-virtualenv curl ntp build-essential screen cmake pkg-config libboost-all-dev libevent-dev libunbound-dev libminiupnpc-dev libunwind8-dev liblzma-dev libldns-dev libexpat1-dev libgtest-dev libzmq3-dev -cd ~ -git clone https://github.com/Snipa22/nodejs-pool.git # Change this depending on how the deployment goes. -cd /usr/src/gtest -sudo cmake . -sudo make -sudo mv libg* /usr/lib/ -cd ~ -sudo systemctl enable ntp + +DEBIAN_FRONTEND=noninteractive apt-get update +DEBIAN_FRONTEND=noninteractive apt-get full-upgrade -y +DEBIAN_FRONTEND=noninteractive apt-get install -y ntp sudo ufw +timedatectl set-timezone Etc/UTC + +adduser --disabled-password --gecos "" user +grep -q "user ALL=(ALL) NOPASSWD:ALL" /etc/sudoers || echo "user ALL=(ALL) NOPASSWD:ALL" >>/etc/sudoers +su user -c "mkdir /home/user/.ssh" +if [ -f "/root/.ssh/authorized_keys" ]; then + mv /root/.ssh/authorized_keys /home/user/.ssh/authorized_keys + chown user:user /home/user/.ssh/authorized_keys + chmod 600 /home/user/.ssh/authorized_keys + sed -i 's/#\?PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config + sed -i 's/#\?PermitRootLogin .\+/PermitRootLogin no/g' /etc/ssh/sshd_config + sed -i 's/#\? .\+/PermitEmptyPasswords no/g' /etc/ssh/sshd_config + service ssh restart +fi + +ufw default deny incoming +ufw default allow outgoing +ufw allow ssh +ufw allow 3333 +ufw allow 5555 +ufw allow 7777 +ufw allow 9000 +ufw --force enable + +cat >/root/.vimrc <<'EOF' +colorscheme desert +set fo-=ro +EOF + +cat >/home/user/.vimrc <<'EOF' +colorscheme desert +set fo-=ro +EOF +chown user:user /home/user/.vimrc + +DEBIAN_FRONTEND=noninteractive apt-get install -y vim git make g++ cmake libssl-dev libunbound-dev libboost-dev libboost-system-dev libboost-date-time-dev libboost-dev libboost-system-dev libboost-date-time-dev libboost-filesystem-dev libboost-thread-dev libboost-chrono-dev libboost-locale-dev libboost-regex-dev libboost-regex-dev libboost-program-options-dev libzmq3-dev cd /usr/local/src -sudo git clone https://github.com/monero-project/monero.git +git clone https://github.com/monero-project/monero.git cd monero -sudo git checkout v0.11.0.0 -curl https://raw.githubusercontent.com/Snipa22/nodejs-pool/master/deployment/monero_daemon.patch | sudo git apply -v -sudo make -j$(nproc) -sudo cp ~/nodejs-pool/deployment/monero.service /lib/systemd/system/ -sudo useradd -m monerodaemon -d /home/monerodaemon -BLOCKCHAIN_DOWNLOAD_DIR=$(sudo -u monerodaemon mktemp -d) -sudo -u monerodaemon wget --limit-rate=50m -O $BLOCKCHAIN_DOWNLOAD_DIR/blockchain.raw https://downloads.getmonero.org/blockchain.raw -sudo -u monerodaemon /usr/local/src/monero/build/release/bin/monero-blockchain-import --input-file $BLOCKCHAIN_DOWNLOAD_DIR/blockchain.raw --batch-size 20000 --database lmdb#fastest --verify off --data-dir /home/monerodaemon/.bitmonero -sudo -u monerodaemon rm -rf $BLOCKCHAIN_DOWNLOAD_DIR -sudo systemctl daemon-reload -sudo systemctl enable monero -sudo systemctl start monero +git checkout v0.18.3.4 +git submodule update --init +USE_SINGLE_BUILDDIR=1 make -j$(nproc) release || USE_SINGLE_BUILDDIR=1 make -j1 release + +cat >/lib/systemd/system/monero.service <<'EOF' +[Unit] +Description=Monero Daemon +After=network.target + +[Service] +ExecStart=/usr/local/src/monero/build/release/bin/monerod --hide-my-port --prune-blockchain --enable-dns-blocklist --no-zmq --out-peers 64 --non-interactive --restricted-rpc --block-notify '/bin/bash /home/user/nodejs-pool/block_notify.sh' +Restart=always +User=monerodaemon +Nice=10 +CPUQuota=400% + +[Install] +WantedBy=multi-user.target +EOF + +useradd -m monerodaemon -d /home/monerodaemon +systemctl daemon-reload +systemctl enable monero +systemctl start monero + +sleep 30 +echo "Please wait until Monero daemon is fully synced" +tail -f /home/monerodaemon/.bitmonero/bitmonero.log 2>/dev/null | grep Synced & +( tail -F -n0 /home/monerodaemon/.bitmonero/bitmonero.log & ) | egrep -q "You are now synchronized with the network" +killall tail 2>/dev/null +echo "Monero daemon is synced" + +(cat <second); - LOG_PRINT_L2("Considering " << sorted_it->second << ", size " << meta.blob_size << ", current block size " << total_size << "/" << max_total_size << ", current coinbase " << print_money(best_coinbase)); diff --git a/deployment/upgrade_monero.bash b/deployment/upgrade_monero.bash old mode 100644 new mode 100755 index b8cc1e2c..62981890 --- a/deployment/upgrade_monero.bash +++ b/deployment/upgrade_monero.bash @@ -2,12 +2,12 @@ echo "This assumes that you have a standard nodejs-pool install, and will patch and update it to the latest stable builds of Monero." sleep 15 echo "Continuing install, this will prompt you for your password if you didn't enable passwordless sudo. Please do not run me as root!" -cd /usr/local/src/monero -sudo git checkout . -sudo git checkout master -sudo git pull -sudo git checkout origin/release-v0.11.0.0 -curl -L https://raw.githubusercontent.com/Snipa22/nodejs-pool/master/deployment/monero_daemon.patch | sudo git apply -v -sudo rm -rf build -sudo make -j$(nproc) -echo "Done building the new Monero daemon! Please go ahead and reboot monero with: sudo systemctl restart monero as soon as the pool source is updated!" +cd /usr/local/src/monero &&\ +sudo git reset --hard origin/master &&\ +sudo git checkout master &&\ +sudo git pull &&\ +sudo git checkout v0.18.3.4 &&\ +sudo git submodule update --force --recursive --init &&\ +sudo rm -rf build &&\ +sudo USE_SINGLE_BUILDDIR=1 nice make release &&\ +echo "Done building the new Monero daemon! Please go ahead and reboot monero with: sudo systemctl restart monero as soon as the pool source is updated!" diff --git a/ex_keys.example.json b/ex_keys.example.json new file mode 100644 index 00000000..0e0c903b --- /dev/null +++ b/ex_keys.example.json @@ -0,0 +1,18 @@ +{ + "CRYPTOPIA": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "CRYPTOPIA_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=", + "TRADEOGRE": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "TRADEOGRE_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "QRYPTOS": "NNNNNN", + "QRYPTOS_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX==", + "LIVECOIN": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "LIVECOIN_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "COINEX": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "COINEX_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "SEVENSEAS": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "SEVENSEAS_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "XEGGEX": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "XEGGEX_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "TXBIT": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX", + "TXBIT_SECRET": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX" +} \ No newline at end of file diff --git a/fix_daemon.sh b/fix_daemon.sh new file mode 100755 index 00000000..2e39c76c --- /dev/null +++ b/fix_daemon.sh @@ -0,0 +1,2 @@ +#!/bin/bash +sudo service monero restart diff --git a/init.js b/init.js index 1a2f2577..b1ec5a06 100644 --- a/init.js +++ b/init.js @@ -80,6 +80,12 @@ global.mysql.query("SELECT * FROM config").then(function (rows) { case 'blockManager': require('./lib/blockManager.js'); break; + case 'altblockManager': + require('./lib2/altblockManager.js'); + break; + case 'altblockExchange': + require('./lib2/altblockExchange.js'); + break; case 'payments': require('./lib/payments.js'); break; @@ -92,6 +98,9 @@ global.mysql.query("SELECT * FROM config").then(function (rows) { case 'worker': require('./lib/worker.js'); break; + case 'pool_stats': + require('./lib/pool_stats.js'); + break; case 'longRunner': require('./lib/longRunner.js'); break; diff --git a/init_mini.js b/init_mini.js new file mode 100644 index 00000000..4cff8d68 --- /dev/null +++ b/init_mini.js @@ -0,0 +1,44 @@ +"use strict"; + +function init(callback) { + + let fs = require("fs"); + let mysql = require("promise-mysql"); + + let config = fs.readFileSync("../config.json"); + let coinConfig = fs.readFileSync("../coinConfig.json"); + let protobuf = require('protocol-buffers'); + + global.support = require("./lib/support.js")(); + global.config = JSON.parse(config); + global.mysql = mysql.createPool(global.config.mysql); + global.protos = protobuf(fs.readFileSync('../lib/data.proto')); + + global.mysql.query("SELECT * FROM config").then(function (rows) { + rows.forEach(function (row){ + if (!global.config.hasOwnProperty(row.module)) global.config[row.module] = {}; + if (global.config[row.module].hasOwnProperty(row.item)) return; + switch(row.item_type){ + case 'int': global.config[row.module][row.item] = parseInt(row.item_value); break; + case 'bool': global.config[row.module][row.item] = (row.item_value === "true"); break; + case 'string': global.config[row.module][row.item] = row.item_value; break; + case 'float': global.config[row.module][row.item] = parseFloat(row.item_value); break; + } + }); + + }).then(function(){ + global.config['coin'] = JSON.parse(coinConfig)[global.config.coin]; + let coinInc = require(global.config.coin.funcFile); + global.coinFuncs = new coinInc(); + let comms = require('./lib/local_comms'); + global.database = new comms(); + global.database.initEnv(); + + }).then(function(){ + callback(); + }); +} + +module.exports = { + init: init +}; \ No newline at end of file diff --git a/lib/api.js b/lib/api.js index e4274bfe..1995a6a8 100644 --- a/lib/api.js +++ b/lib/api.js @@ -1,18 +1,20 @@ "use strict"; const express = require('express'); // call express +const apicache = require('apicache'); const app = express(); // define our app using express +const cache = apicache.middleware; const server = require('http').createServer(app); const cluster = require('cluster'); const async = require("async"); const debug = require("debug")("api"); const btcValidator = require('wallet-address-validator'); -const cnUtil = require('cryptonote-util'); +const cnUtil = require('cryptoforknote-util'); const bodyParser = require('body-parser'); const jwt = require('jsonwebtoken'); // used to create, sign, and verify tokens const crypto = require('crypto'); const cors = require('cors'); -let addressBase58Prefix = cnUtil.address_decode(new Buffer(global.config.pool.address)); +let addressBase58Prefix = cnUtil.address_decode(Buffer.from(global.config.pool.address)); let threadName = ""; if (cluster.isMaster) { @@ -22,64 +24,73 @@ if (cluster.isMaster) { } let pool_list = []; -if(global.config.pplns.enable === true){ - pool_list.push('pplns'); -} -if(global.config.pps.enable === true){ - pool_list.push('pps'); -} -if(global.config.solo.enable === true){ - pool_list.push('solo'); -} - -app.use(cors({origin: true})); +if (global.config.pplns.enable === true) pool_list.push('pplns'); +if (global.config.pps.enable === true) pool_list.push('pps'); +if (global.config.solo.enable === true) pool_list.push('solo'); + +let RPM = 0; + +setInterval(function () { + console.log(threadName + "RPM: " + RPM); + RPM = 0; +}, 60*1000); + +//var whitelist = ['https://moneroocean.stream', 'https://test.moneroocean.stream']; +//app.use(cors({ +// origin: function (origin, callback) { return callback(null, whitelist.indexOf(origin) !== -1 || !origin); } +//})); +app.use(cors()); app.use(bodyParser.urlencoded({extended: false})); app.use(bodyParser.json()); +function get_identifiers(address) { + return global.database.getCache('identifiers:' + address); +} + // Support Functions that are reused now function getAllWorkerHashCharts(address, callback){ - let identifiers = global.database.getCache(address + '_identifiers'); - let returnData = {global: global.database.getCache(address)['hashHistory']}; - if (identifiers !== false){ - identifiers.sort(); - } else { - return returnData; - } + let identifiers = get_identifiers(address); + let returnData = { global: global.database.getCache("history:" + address)['hashHistory'] }; + if (identifiers === false || identifiers.length == 0) return callback(null, returnData); let intCounter = 0; - identifiers.forEach(function(identifier){ - returnData[identifier] = global.database.getCache(address+"_"+identifier)['hashHistory']; - intCounter += 1; - if (intCounter === identifiers.length){ - return callback(null, returnData); - } + identifiers.sort().forEach(function(identifier){ + returnData[identifier] = global.database.getCache("history:" + address + "_" + identifier)['hashHistory']; + if (++ intCounter === identifiers.length) return callback(null, returnData); }); } function getAllWorkerStats(address, callback){ - let identifiers = global.database.getCache(address + '_identifiers'); + let identifiers = get_identifiers(address); let globalCache = global.database.getCache(address); - let returnData = {global: { - lts: Math.floor(globalCache.lastHash / 1000), - identifer: 'global', - hash: globalCache.hash, - totalHash: globalCache.totalHashes - }}; + let globalStatsCache = global.database.getCache("stats:" + address); + let returnData = { + global: { + lts: globalStatsCache !== false ? Math.floor(globalStatsCache.lastHash / 1000) : false, + identifer: 'global', + hash: globalStatsCache !== false ? globalStatsCache.hash : false, + hash2: globalStatsCache !== false ? globalStatsCache.hash2 : false, + totalHash: globalCache !== false ? globalCache.totalHashes : false, + validShares: globalCache !== false ? Number(globalCache.goodShares) : false, + invalidShares: globalCache !== false ? (globalCache.badShares ? Number(globalCache.badShares) : 0) : false + + } + }; + if (identifiers === false || identifiers.length == 0) return callback(null, returnData); let intCounter = 0; - if (identifiers === false){ - return callback(null, returnData); - } identifiers.sort().forEach(function(identifier){ - let cachedData = global.database.getCache(address+"_"+identifier); + let id2 = address + "_" + identifier; + let cachedData = global.database.getCache(id2); + let cachedStatsData = global.database.getCache("stats:" + id2); returnData[identifier] = { - lts: Math.floor(cachedData.lastHash / 1000), - identifer: identifier, - hash: cachedData.hash, - totalHash: cachedData.totalHashes + lts: cachedStatsData !== false ? Math.floor(cachedStatsData.lastHash / 1000) : false, + identifer: identifier, + hash: cachedStatsData !== false ? cachedStatsData.hash : false, + hash2: cachedStatsData !== false ? cachedStatsData.hash2 : false, + totalHash: cachedData !== false ? cachedData.totalHashes : false, + validShares: cachedData !== false ? Number(cachedData.goodShares) : false, + invalidShares: cachedData !== false ? (cachedData.badShares ? Number(cachedData.badShares) : 0) : false }; - intCounter += 1; - if (intCounter === identifiers.length){ - return callback(null, returnData); - } + if (++ intCounter === identifiers.length) return callback(null, returnData); }); } @@ -88,6 +99,7 @@ function getAddressStats(address, extCallback){ let address_pt = address_parts[0]; let payment_id = address_parts[1]; let cachedData = global.database.getCache(address); + let cachedStatsData = global.database.getCache("stats:" + address); let paidQuery = "SELECT SUM(amount) as amt FROM payments WHERE payment_address = ? AND payment_id = ?"; let txnQuery = "SELECT count(id) as amt FROM payments WHERE payment_address = ? AND payment_id = ?"; let unpaidQuery = "SELECT SUM(amount) as amt FROM balance WHERE payment_address = ? AND payment_id = ?"; @@ -99,8 +111,15 @@ function getAddressStats(address, extCallback){ async.waterfall([ function (callback) { debug(threadName + "Checking Influx for last 10min avg for /miner/address/stats"); - return callback(null, {hash: cachedData.hash, identifier: 'global', lastHash: Math.floor(cachedData.lastHash / 1000), - totalHashes: cachedData.totalHashes, validShares: Number(cachedData.goodShares), invalidShares: Number(cachedData.badShares)}); + return callback(null, { + hash: cachedStatsData.hash, + hash2: cachedStatsData.hash2, + identifier: 'global', + lastHash: Math.floor(cachedStatsData.lastHash / 1000), + totalHashes: cachedData.totalHashes, + validShares: Number(cachedData.goodShares), + invalidShares: cachedData.badShares ? Number(cachedData.badShares) : 0 + }); }, function (returnData, callback) { debug(threadName + "Checking MySQL total amount paid for /miner/address/stats"); @@ -114,6 +133,9 @@ function getAddressStats(address, extCallback){ } } return callback(null, returnData); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(null, {}); }); }, function (returnData, callback) { @@ -128,6 +150,9 @@ function getAddressStats(address, extCallback){ } } return callback(null, returnData); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(null, {}); }); }, function (returnData, callback) { @@ -142,6 +167,9 @@ function getAddressStats(address, extCallback){ } } return callback(true, returnData); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(null, {}); }); } ], function (err, result) { @@ -162,7 +190,8 @@ function getAddressStats(address, extCallback){ // test route to make sure everything is working (accessed at GET http://localhost:8080/api) // Config API -app.get('/config', function (req, res) { +app.get('/config', cache('5 minutes'), function (req, res) { + ++ RPM; res.json({ pplns_fee: global.config.payout.pplnsFee, pps_fee: global.config.payout.ppsFee, @@ -174,23 +203,30 @@ app.get('/config', function (req, res) { dev_donation: global.config.payout.devDonation, pool_dev_donation: global.config.payout.poolDevDonation, maturity_depth: global.config.payout.blocksRequired, - min_denom: global.config.payout.denom * global.config.general.sigDivisor + min_denom: global.config.payout.denom * global.config.general.sigDivisor, + coin_code: global.config.general.coinCode }); }); // Pool APIs -app.get('/pool/address_type/:address', function (req, res) { +app.get('/pool/address_type/:address', cache('10 seconds'), function (req, res) { + ++ RPM; let address = req.params.address; - if (addressBase58Prefix === cnUtil.address_decode(new Buffer(address))) { - res.json({valid: true, address_type: 'XMR'}); - } else if (btcValidator.validate(this.address) && global.config.general.allowBitcoin) { - res.json({valid: true, address_type: 'BTC'}); + if (addressBase58Prefix === cnUtil.address_decode(Buffer.from(address))) { + res.json({valid: true, address_type: global.config.general.coinCode}); } else { res.json({valid: false}); } }); -app.get('/pool/stats', function (req, res) { +app.get('/pool/motd', cors(), cache('60 seconds'), function (req, res) { + ++ RPM; + const news = global.database.getCache('news'); + res.json({created: news.created, subject: news.subject, body: news.body}); +}); + +app.get('/pool/stats', cors(), cache('10 seconds'), function (req, res) { + ++ RPM; let localCache = global.database.getCache('pool_stats_global'); delete(localCache.minerHistory); delete(localCache.hashHistory); @@ -198,15 +234,18 @@ app.get('/pool/stats', function (req, res) { res.json({pool_list: pool_list, pool_statistics: localCache, last_payment: !lastPayment ? 0 : lastPayment}); }); -app.get('/pool/chart/hashrate', function (req, res) { +app.get('/pool/chart/hashrate', cache('10 seconds'), function (req, res) { + ++ RPM; res.json(global.database.getCache('global_stats')['hashHistory']); }); -app.get('/pool/chart/miners', function (req, res) { +app.get('/pool/chart/miners', cache('10 seconds'), function (req, res) { + ++ RPM; res.json(global.database.getCache('global_stats')['minerHistory']); }); -app.get('/pool/chart/hashrate/:pool_type', function (req, res) { +app.get('/pool/chart/hashrate/:pool_type', cache('10 seconds'), function (req, res) { + ++ RPM; let pool_type = req.params.pool_type; let localCache; switch (pool_type) { @@ -219,13 +258,14 @@ app.get('/pool/chart/hashrate/:pool_type', function (req, res) { case 'solo': localCache = global.database.getCache('solo_stats'); break; - case 'default': + default: return res.json({'error': 'Invalid pool type'}); } res.json(localCache['hashHistory']); }); -app.get('/pool/chart/miners/:pool_type', function (req, res) { +app.get('/pool/chart/miners/:pool_type', cache('10 seconds'), function (req, res) { + ++ RPM; let pool_type = req.params.pool_type; let localCache; switch (pool_type) { @@ -238,13 +278,14 @@ app.get('/pool/chart/miners/:pool_type', function (req, res) { case 'solo': localCache = global.database.getCache('stats_solo'); break; - case 'default': + default: return res.json({'error': 'Invalid pool type'}); } res.json(localCache['minerHistory']); }); -app.get('/pool/stats/:pool_type', function (req, res) { +app.get('/pool/stats/:pool_type', cache('10 seconds'), function (req, res) { + ++ RPM; let pool_type = req.params.pool_type; let localCache; switch (pool_type) { @@ -260,7 +301,7 @@ app.get('/pool/stats/:pool_type', function (req, res) { localCache = global.database.getCache('pool_stats_solo'); localCache.fee = global.config.payout.soloFee; break; - case 'default': + default: return res.json({'error': 'Invalid pool type'}); } delete(localCache.minerHistory); @@ -268,43 +309,62 @@ app.get('/pool/stats/:pool_type', function (req, res) { res.json({pool_statistics: localCache}); }); -app.get('/pool/ports', function (req, res) { +app.get('/pool/ports', cache('10 seconds'), function (req, res) { + ++ RPM; res.json(global.database.getCache('poolPorts')); }); -app.get('/pool/blocks/:pool_type', function (req, res) { +app.get('/pool/blocks/:pool_type', cache('10 seconds'), function (req, res) { + ++ RPM; + let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; + let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; + res.json(global.database.getBlockList(req.params.pool_type, page*limit, (page + 1) * limit)); +}); + +app.get('/pool/altblocks/:pool_type', cache('10 seconds'), function (req, res) { + ++ RPM; let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; - res.json(global.database.getBlockList(req.params.pool_type).slice(page*limit, (page + 1) * limit)); + res.json(global.database.getAltBlockList(req.params.pool_type, null, page*limit, (page + 1) * limit)); }); -app.get('/pool/blocks', function (req, res) { +app.get('/pool/blocks', cache('10 seconds'), function (req, res) { + ++ RPM; let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; - res.json(global.database.getBlockList().slice(page*limit, (page + 1) * limit)); + res.json(global.database.getBlockList(null, page*limit, (page + 1) * limit)); }); -app.get('/pool/payments/:pool_type', function (req, res) { +app.get('/pool/altblocks', cache('10 seconds'), function (req, res) { + ++ RPM; + let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; + let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; + res.json(global.database.getAltBlockList(null, null, page*limit, (page + 1) * limit)); +}); + +app.get('/pool/coin_altblocks/:coin_port', cache('10 seconds'), function (req, res) { + ++ RPM; + let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; + let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; + res.json(global.database.getAltBlockList(null, parseInt(req.params.coin_port), page*limit, (page + 1) * limit)); +}); + +app.get('/pool/payments/:pool_type', cache('1 minute'), function (req, res) { + ++ RPM; let pool_type = req.params.pool_type; let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 10; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; switch (pool_type) { - case 'pplns': - break; - case 'pps': - break; - case 'solo': - break; - case 'default': - return res.json({'error': 'Invalid pool type'}); + case 'pplns': break; + case 'pps': break; + case 'solo': break; + default: return res.json({'error': 'Invalid pool type'}); } let paymentIds = []; let query = "SELECT distinct(transaction_id) as txnID FROM payments WHERE pool_type = ? ORDER BY transaction_id DESC LIMIT ? OFFSET ?"; let response = []; global.mysql.query(query, [pool_type, limit, page * limit]).then(function (rows) { - if (rows.length === 0) { - return res.json([]); - } + if (rows.length === 0) return res.json([]); rows.forEach(function (row, index, array) { paymentIds.push(row.txnID); if (array.length === paymentIds.length) { @@ -324,6 +384,9 @@ app.get('/pool/payments/:pool_type', function (req, res) { return res.json(response.sort(global.support.tsCompare)); } }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); }); } }); @@ -333,17 +396,19 @@ app.get('/pool/payments/:pool_type', function (req, res) { }); }); -app.get('/pool/payments', function (req, res) { +// cache pool_type here just to avoid multiple SQL requests +let tx_pool_types = {}; + +app.get('/pool/payments', cache('1 minute'), function (req, res) { + ++ RPM; let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 10; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; let query = "SELECT * FROM transactions ORDER BY id DESC LIMIT ? OFFSET ?"; global.mysql.query(query, [limit, page * limit]).then(function (rows) { - if (rows.length === 0) { - return res.json([]); - } + if (rows.length === 0) return res.json([]); let response = []; rows.forEach(function (row, index, array) { - global.mysql.query("SELECT pool_type FROM payments WHERE transaction_id = ? LIMIT 1", [row.id]).then(function (ptRows) { + if (pool_list.length === 1 || (row.id in tx_pool_types && tx_pool_types[row.id] != "?")) { let ts = new Date(row.submitted_time); response.push({ id: row.id, @@ -353,12 +418,37 @@ app.get('/pool/payments', function (req, res) { fee: row.fees, value: row.xmr_amt, ts: ts.getTime(), - pool_type: ptRows[0].pool_type + pool_type: (pool_list.length === 1 ? pool_list[0] : tx_pool_types[row.id]) }); if (array.length === response.length) { res.json(response.sort(global.support.tsCompare)); } - }); + } else { + global.mysql.query("SELECT pool_type FROM payments WHERE transaction_id = ? LIMIT 1", [row.id]).then(function (ptRows) { + if (ptRows.length === 0) { + console.error("Unknown pool_type for tx_id " + row.id); + ptRows = [ { "pool_type": "?" } ]; + } + let ts = new Date(row.submitted_time); + tx_pool_types[row.id] = ptRows[0].pool_type; + response.push({ + id: row.id, + hash: row.transaction_hash, + mixins: row.mixin, + payees: row.payees, + fee: row.fees, + value: row.xmr_amt, + ts: ts.getTime(), + pool_type: ptRows[0].pool_type + }); + if (array.length === response.length) { + res.json(response.sort(global.support.tsCompare)); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); + } }); }).catch(function (err) { console.error(threadName + "Error getting miner payments: " + JSON.stringify(err)); @@ -367,17 +457,20 @@ app.get('/pool/payments', function (req, res) { }); // Network APIs -app.get('/network/stats', function (req, res) { +app.get('/network/stats', cache('10 seconds'), function (req, res) { + ++ RPM; res.json(global.database.getCache('networkBlockInfo')); }); // Miner APIs -app.get('/miner/:address/identifiers', function (req, res) { +app.get('/miner/:address/identifiers', cache('10 seconds'), function (req, res) { + ++ RPM; let address = req.params.address; - return res.json(global.database.getCache(address + '_identifiers')); + return res.json(get_identifiers(address)); }); -app.get('/miner/:address/payments', function (req, res) { +app.get('/miner/:address/payments', cache('1 minute'), function (req, res) { + ++ RPM; let limit = typeof(req.query.limit) !== 'undefined' ? Number(req.query.limit) : 25; let page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; let address_parts = req.params.address.split('.'); @@ -391,9 +484,7 @@ app.get('/miner/:address/payments', function (req, res) { } let response = []; global.mysql.query(query, [address, payment_id, limit, page * limit]).then(function (rows) { - if (rows.length === 0) { - return res.json(response); - } + if (rows.length === 0) return res.json(response); rows.forEach(function (row, index, array) { debug(threadName + "Got rows from initial SQL query: " + JSON.stringify(row)); global.mysql.query("SELECT transaction_hash, mixin FROM transactions WHERE id = ? ORDER BY id DESC", [row.transaction_id]).then(function (txnrows) { @@ -410,6 +501,9 @@ app.get('/miner/:address/payments', function (req, res) { return res.json(response.sort(global.support.tsCompare)); } }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); }); }); }).catch(function (err) { @@ -418,13 +512,59 @@ app.get('/miner/:address/payments', function (req, res) { }); }); -app.get('/miner/:address/stats/allWorkers', function (req, res) { +app.get('/miner/:address/block_payments', cache('1 minute'), function (req, res) { + ++ RPM; + const limit = typeof(req.query.limit) !== 'undefined' ? (Number(req.query.limit) > 100 ? 100 : Number(req.query.limit)) : 10; + const page = typeof(req.query.page) !== 'undefined' ? Number(req.query.page) : 0; + + const address_parts = req.params.address.split('.'); + const address = address_parts[0]; + const payment_id = address_parts[1]; + const where_str = typeof(payment_id) === 'undefined' + ? "payment_address = '" + address + "' AND (payment_id IS NULL OR payment_id = '')" + : "payment_address = '" + address + "' AND payment_id = '" + payment_id + "'"; + + global.mysql.query("SELECT * FROM paid_blocks WHERE paid_time > (NOW() - INTERVAL 7 DAY) ORDER BY id DESC LIMIT ? OFFSET ?", [limit, page * limit]).then(function (rows) { + if (rows.length === 0) return res.json([]); + let block_hexes = []; + rows.forEach(function (row) { block_hexes.push('"' + row.hex + '"'); }); + + global.mysql.query("SELECT hex, amount FROM block_balance WHERE " + where_str + " AND hex IN (" + block_hexes.join() + ")").then(function (rows2) { + let block_miner_shares = {}; + rows2.forEach(function (row2) { block_miner_shares[row2.hex] = row2.amount; }); + let response = []; + rows.forEach(function (row) { + const miner_payment_share = row.hex in block_miner_shares ? block_miner_shares[row.hex] : 0; + response.push({ + id: row.id, + ts: (new Date(row.paid_time)).getTime() / 1000, + ts_found: (new Date(row.found_time)).getTime() / 1000, + port: row.port, + hash: row.hex, + value_percent: miner_payment_share * 100.0, + value: miner_payment_share * row.amount / global.config.general.sigDivisor + }); + }); + return res.json(response.sort(global.support.tsCompare)); + }).catch(function (err) { + console.error(threadName + "Error getting block_balance miner block payments: " + JSON.stringify(err)); + return res.json({error: 'Issue getting block payments'}); + }); + }).catch(function (err) { + console.error(threadName + "Error getting paid_blocks miner block payments: " + JSON.stringify(err)); + return res.json({error: 'Issue getting block payments'}); + }); +}); + +app.get('/miner/:address/stats/allWorkers', cache('10 seconds'), function (req, res) { + ++ RPM; getAllWorkerStats(req.params.address, function(err, data){ return res.json(data); }); }); -app.get('/miner/:address/stats/:identifier', function (req, res) { +app.get('/miner/:address/stats/:identifier', cache('10 seconds'), function (req, res) { + ++ RPM; let address = req.params.address; let identifier = req.params.identifier; let memcKey = address + "_" + identifier; @@ -434,57 +574,179 @@ app.get('/miner/:address/stats/:identifier', function (req, res) { lastHash: localTimes.miners[miner] */ let cachedData = global.database.getCache(memcKey); + let cachedStatsData = global.database.getCache("stats:" + memcKey); return res.json({ - lts: Math.floor(cachedData.lastHash / 1000), - identifer: identifier, - hash: cachedData.hash, - totalHash: cachedData.totalHashes, - validShares: Number(cachedData.goodShares), - invalidShares: Number(cachedData.badShares) + lts: Math.floor(cachedStatsData.lastHash / 1000), + identifer: identifier, + hash: cachedStatsData.hash, + hash2: cachedStatsData.hash2, + totalHash: cachedData.totalHashes, + validShares: Number(cachedData.goodShares), + invalidShares: cachedData.badShares ? Number(cachedData.badShares) : 0 }); }); -app.get('/miner/:address/chart/hashrate', function (req, res) { - return res.json(global.database.getCache(req.params.address)['hashHistory']); +app.get('/miner/:address/chart/hashrate', cache('10 seconds'), function (req, res) { + ++ RPM; + return res.json(global.database.getCache("history:" + req.params.address)['hashHistory']); }); -app.get('/miner/:address/chart/hashrate/allWorkers', function (req, res) { +app.get('/miner/:address/chart/hashrate/allWorkers', cache('10 seconds'), function (req, res) { + ++ RPM; getAllWorkerHashCharts(req.params.address, function(err, data){ return res.json(data); }); }); -app.get('/miner/:address/chart/hashrate/:identifier', function (req, res) { - return res.json(global.database.getCache(req.params.address + "_" + req.params.identifier)['hashHistory']); +app.get('/miner/:address/chart/hashrate/:identifier', cache('10 seconds'), function (req, res) { + ++ RPM; + return res.json(global.database.getCache("history:" + req.params.address + "_" + req.params.identifier)['hashHistory']); }); -app.get('/miner/:address/stats', function (req, res) { +app.get('/miner/:address/stats', cache('1 minute'), function (req, res) { + ++ RPM; getAddressStats(req.params.address, function(err, data){ return res.json(data); }); }); +app.get('/user/:address', function (req, res) { + ++ RPM; + global.mysql.query("SELECT payout_threshold, enable_email FROM users WHERE username = ? LIMIT 1", [req.params.address]).then(function(row){ + if (row.length == 1) { + return res.json({payout_threshold: row[0].payout_threshold, email_enabled: row[0].enable_email}); + } else { + return res.json({payout_threshold: global.support.decimalToCoin(global.config.payout.defaultPay), email_enabled: 0}); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); +}); + +app.post('/user/subscribeEmail', function (req, res) { + ++ RPM; + const username = req.body.username; + if (!username) return res.status(401).send({'success': false, 'msg': "No \"username\" parameter was found"}); + if (!("enabled" in req.body)) return res.status(401).send({'success': false, 'msg': "No \"enabled\" parameter was found"}); + if (!("from" in req.body)) return res.status(401).send({'success': false, 'msg': "No \"from\" parameter was found"}); + if (!("to" in req.body)) return res.status(401).send({'success': false, 'msg': "No \"to\" parameter was found"}); + const enabled = req.body.enabled; + const from = req.body.from; + const to = req.body.to; + if (from === "" && to === "") { + global.mysql.query("UPDATE users SET enable_email = ? WHERE username = ?", [enabled, username]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + return res.status(401).json({'error': 'This XMR address does not have email subscription'}); + } else { + return res.json({'msg': 'Email preferences were updated'}); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); + } else if (from === "") { + global.mysql.query("UPDATE users SET enable_email = ?, email = ? WHERE username = ? AND (email IS NULL OR email = '')", [enabled, to, username]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + if (global.database.getCache(username) === false) return res.status(401).send({'success': false, 'msg': "Can't set email for unknown user"}); + global.mysql.query("INSERT INTO users (username, enable_email, email) VALUES (?, ?, ?)", [username, enabled, to]).then(function () { + return res.json({'msg': 'Email preferences were updated'}); + }).catch(function(err) { + return res.status(401).json({'error': 'Please specify valid FROM email'}); + }); + } else { + return res.json({'msg': 'Email preferences were updated'}); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); + } else { + global.mysql.query("UPDATE users SET enable_email = ?, email = ? WHERE username = ? AND email = ?", [enabled, to, username, from]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + return res.status(401).json({'error': 'FROM email does not match'}); + } else { + return res.json({'msg': 'Email preferences were updated'}); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); + } +}); + +app.get('/user/:address/unsubscribeEmail', function (req, res) { + ++ RPM; + global.mysql.query("UPDATE users SET enable_email = 0 WHERE username = ?", [req.params.address]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + return res.status(401).json({'error': 'This XMR address does not have email subscription'}); + } else { + return res.json({'msg': 'Your email was unsubscribed from further notifications'}); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); +}); + +app.post('/user/updateThreshold', function (req, res) { + ++ RPM; + let threshold = req.body.threshold; + if (!threshold) return res.status(401).send({'success': false, 'msg': "Can't set threshold to a wrong value"}); + if (threshold > 1000) threshold = 1000; + const username = req.body.username; + if (!username || global.database.getCache(username) === false) return res.status(401).send({'success': false, 'msg': "Can't set threshold for unknown user"}); + const threshold2 = global.support.decimalToCoin(threshold < global.config.payout.walletMin ? global.config.payout.walletMin : threshold); + global.mysql.query("SELECT * FROM users WHERE username = ? AND payout_threshold_lock = '1'", [username]).then(function (rows) { + if (rows.length === 0) { + global.mysql.query("INSERT INTO users (username, payout_threshold) VALUES (?, ?) ON DUPLICATE KEY UPDATE payout_threshold=?", [username, threshold2, threshold2]).then(function () { + return res.json({'msg': 'Threshold updated, set to: ' + global.support.coinToDecimal(threshold2)}); + }); + } else { + return res.status(401).send({'success': false, 'msg':"Can't update locked payment threshold"}); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); +}); + // Authentication app.post('/authenticate', function (req, res) { + ++ RPM; let hmac; try{ hmac = crypto.createHmac('sha256', global.config.api.secKey).update(req.body.password).digest('hex'); } catch (e) { - return res.status(401).send({'success': false, msg: 'Invalid username/password'}); + return res.status(401).send({'success': false, msg: 'Invalid password'}); } global.mysql.query("SELECT * FROM users WHERE username = ? AND ((pass IS null AND email = ?) OR (pass = ?))", [req.body.username, req.body.password, hmac]).then(function (rows) { if (rows.length === 0) { - return res.status(401).send({'success': false, msg: 'Invalid username/password'}); + global.mysql.query("SELECT * FROM users WHERE username = ?", [req.body.username]).then(function (rows) { + if (rows.length === 0) { + return res.status(401).send({'success': false, msg: 'Password is not set, so you can not login now.'}); + } + global.mysql.query("SELECT * FROM users WHERE username = ? AND pass IS null", [req.body.username]).then(function (rows) { + if (rows.length !== 0) { + return res.status(401).send({'success': false, msg: 'Wrong password. Password equals to string after : character in your miner password field.'}); + } + return res.status(401).send({'success': false, msg: 'Wrong password. Password was set by you in Dashboard Options before.'}); + }); + }); + } else { + let token = jwt.sign({id: rows[0].id, admin: rows[0].admin}, global.config.api.secKey, {expiresIn: '1d'}); + return res.json({'success': true, 'msg': token}); } - let token = jwt.sign({id: rows[0].id, admin: rows[0].admin}, global.config.api.secKey, {expiresIn: '1d'}); - return res.json({'success': true, 'msg': token}); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); }); }); // JWT Verification // get an instance of the router for api routes let secureRoutes = express.Router(); -let adminRoutes = express.Router(); +//let adminRoutes = express.Router(); // route middleware to verify a token secureRoutes.use(function (req, res, next) { @@ -510,315 +772,68 @@ secureRoutes.use(function (req, res, next) { // Secure/logged in routes. secureRoutes.get('/tokenRefresh', function (req, res) { + ++ RPM; let token = jwt.sign({id: req.decoded.id, admin: req.decoded.admin}, global.config.api.secKey, {expiresIn: '1d'}); return res.json({'msg': token}); }); secureRoutes.get('/', function (req, res) { - global.mysql.query("SELECT payout_threshold, enable_email FROM users WHERE id = ?", [req.decoded.id]).then(function(row){ - return res.json({msg: {payout_threshold: row[0].payout_threshold, email_enabled: row[0].enable_email}}); + ++ RPM; + global.mysql.query("SELECT payout_threshold, enable_email, email FROM users WHERE id = ?", [req.decoded.id]).then(function(row){ + return res.json({msg: {payout_threshold: row[0].payout_threshold, email_enabled: row[0].enable_email, email: row[0].email}}); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); }); }); secureRoutes.post('/changePassword', function (req, res) { + ++ RPM; let hmac = crypto.createHmac('sha256', global.config.api.secKey).update(req.body.password).digest('hex'); global.mysql.query("UPDATE users SET pass = ? WHERE id = ?", [hmac, req.decoded.id]).then(function () { return res.json({'msg': 'Password updated'}); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); + }); +}); + +secureRoutes.post('/changeEmail', function (req, res) { + ++ RPM; + global.mysql.query("UPDATE users SET email = ? WHERE id = ?", [req.body.email, req.decoded.id]).then(function () { + return res.json({'msg': 'Updated email was set to: ' + req.body.email}); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); }); }); secureRoutes.post('/toggleEmail', function (req, res) { + ++ RPM; global.mysql.query("UPDATE users SET enable_email = NOT enable_email WHERE id = ?", [req.decoded.id]).then(function () { return res.json({'msg': 'Email toggled'}); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); }); }); secureRoutes.post('/changePayoutThreshold', function (req, res) { + ++ RPM; let threshold = req.body.threshold; - if (threshold < global.config.payout.walletMin) { - threshold = global.config.payout.walletMin; - } + if (!threshold) return res.status(401).send({'success': false, 'msg': "Can't set threshold to a wrong value"}); + if (threshold < global.config.payout.walletMin) threshold = global.config.payout.walletMin; threshold = global.support.decimalToCoin(threshold); global.mysql.query("UPDATE users SET payout_threshold = ? WHERE id = ?", [threshold, req.decoded.id]).then(function () { return res.json({'msg': 'Threshold updated, set to: ' + global.support.coinToDecimal(threshold)}); - }); -}); - -// Administrative routes/APIs - -adminRoutes.use(function (req, res, next) { - let token = req.body.token || req.query.token || req.headers['x-access-token']; - if (token) { - jwt.verify(token, global.config.api.secKey, function (err, decoded) { - if (decoded.admin !== 1) { - return res.status(403).send({ - success: false, - msg: 'You are not an admin.' - }); - } - if (err) { - return res.json({success: false, msg: 'Failed to authenticate token.'}); - } else { - req.decoded = decoded; - next(); - } - }); - - } else { - return res.status(403).send({ - success: false, - msg: 'No token provided.' - }); - } -}); - -adminRoutes.get('/stats', function (req, res) { - /* - Admin interface stats. - For each pool type + global, we need the following: - Total Owed, Total Paid, Total Mined, Total Blocks, Average Luck - */ - let intCache = { - 'pplns': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'pps': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'solo': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'global': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0}, - 'fees': {owed: 0, paid: 0, mined: 0, shares: 0, targetShares: 0} - }; - async.series([ - function (callback) { - global.mysql.query("select * from balance").then(function (rows) { - rows.forEach(function (row) { - intCache[row.pool_type].owed += row.amount; - intCache.global.owed += row.amount; - }); - }).then(function () { - return callback(null); - }); - }, - function (callback) { - global.mysql.query("select * from payments").then(function (rows) { - rows.forEach(function (row) { - intCache[row.pool_type].paid += row.amount; - intCache.global.paid += row.amount; - }); - }).then(function () { - return callback(null); - }); - }, - function (callback) { - global.database.getBlockList().forEach(function (block) { - intCache[block.pool_type].mined += block.value; - intCache.global.mined += block.value; - intCache[block.pool_type].shares += block.shares; - intCache.global.shares += block.shares; - intCache[block.pool_type].targetShares += block.diff; - intCache.global.targetShares += block.diff; - }); - return callback(null); - } - ], function () { - return res.json(intCache); - }); -}); - -adminRoutes.get('/wallet', function (req, res) { - // Stats for the admin interface. - // Load the wallet state from cache, NOTHING HAS DIRECT ACCESS. - // walletStateInfo - return res.json(global.database.getCache('walletStateInfo')); -}); - -adminRoutes.get('/wallet/history', function (req, res) { - // walletHistory - if (req.decoded.admin === 1) { - return res.json(global.database.getCache('walletHistory')); - } -}); - -adminRoutes.get('/ports', function (req, res) { - let retVal = []; - global.mysql.query("SELECT * FROM port_config").then(function (rows) { - rows.forEach(function (row) { - retVal.push({ - port: row.poolPort, - diff: row.difficulty, - desc: row.portDesc, - portType: row.portType, - hidden: row.hidden === 1, - ssl: row.ssl === 1 - }); - }); - }).then(function () { - return res.json(retVal); - }); -}); - -adminRoutes.post('/ports', function (req, res) { - global.mysql.query("SELECT * FROM port_config WHERE poolPort = ?", [req.body.port]).then(function (rows) { - if (rows.length !== 0) { - return "Port already exists with that port number."; - } - if (req.body.diff > global.config.pool.maxDifficulty || req.body.diff < global.config.pool.minDifficulty) { - return "Invalid difficulty."; - } - if (["pplns", "solo", "pps"].indexOf(req.body.portType) === -1) { - return "Invalid port type"; - } - global.mysql.query("INSERT INTO port_config (poolPort, difficulty, portDesc, portType, hidden, ssl) VALUES (?, ?, ?, ?, ?, ?)", - [req.body.port, req.body.diff, req.body.desc, req.body.portType, req.body.hidden === 1, req.body.ssl === 1]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Added port to database"}); - }); -}); - -adminRoutes.put('/ports', function (req, res) { - let portNumber = Number(req.body.portNum); - global.mysql.query("SELECT * FROM port_config WHERE poolPort = ?", [portNumber]).then(function (rows) { - if (rows.length === 0) { - return "Port doesn't exist in the database"; - } - if (req.body.diff > global.config.pool.maxDifficulty || req.body.diff < global.config.pool.minDifficulty) { - return "Invalid difficulty."; - } - if (["pplns", "solo", "pps"].indexOf(req.body.portType) === -1) { - return "Invalid port type"; - } - global.mysql.query("UPDATE port_config SET difficulty=?, portDesc=?, portType=?, hidden=?, ssl=? WHERE poolPort = ?", - [req.body.diff, req.body.desc, req.body.portType, req.body.hidden === 1, req.body.ssl === 1, portNumber]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Updated port in database"}); - }); -}); - -adminRoutes.delete('/ports', function (req, res) { - let portNumber = Number(req.body.portNum); - global.mysql.query("SELECT * FROM port_config WHERE poolPort = ?", [portNumber]).then(function (rows) { - if (rows.length === 0) { - return "Port doesn't exist in the database"; - } - global.mysql.query("DELETE FROM port_config WHERE poolPort = ?", [portNumber]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Added port to database"}); - }); -}); - -adminRoutes.get('/config', function (req, res) { - let retVal = []; - global.mysql.query("SELECT * FROM config").then(function (rows) { - rows.forEach(function (row) { - retVal.push({ - id: row.id, - module: row.module, - item: row.item, - value: row.item_value, - type: row.item_type, - desc: row.item_desc - }); - }); - }).then(function () { - return res.json(retVal); - }); -}); - -adminRoutes.put('/config', function (req, res) { - let configID = Number(req.body.id); - global.mysql.query("SELECT * FROM config WHERE id = ?", [configID]).then(function (rows) { - if (rows.length === 0) { - return "Config item doesn't exist in the database"; - } - global.mysql.query("UPDATE config SET item_value=? WHERE id = ?", [req.body.value, configID]); - }).then(function (err) { - if (typeof(err) === 'string') { - return res.json({success: false, msg: err}); - } - return res.json({success: true, msg: "Updated port in database"}); - }); -}); - -adminRoutes.get('/userList', function (req, res) { - /* - List of all the users in the system. - Might as well do it all, right? :3 - Data Format to be documented. - */ - let intCache = {}; - global.mysql.query("select sum(balance.amount) as amt_due, sum(payments.amount) as amt_paid," + - "balance.payment_address as address, balance.payment_id as payment_id from balance LEFT JOIN payments on " + - "payments.payment_address=balance.payment_address or payments.payment_id=balance.payment_id " + - "group by address, payment_id").then(function (rows) { - rows.forEach(function (row) { - let key = row.address; - if (row.payment_id !== null) { - key += '.' + row.payment_id; - } - intCache[key] = { - paid: row.amt_paid, - due: row.amt_due, - address: key, - workers: [], - lastHash: 0, - totalHashes: 0, - hashRate: 0, - goodShares: 0, - badShares: 0 - }; - }); - }).then(function () { - let minerList = global.database.getCache('minerList'); - if (minerList) { - minerList.forEach(function (miner) { - let minerData = miner.split('_'); - let minerCache = global.database.getCache(miner); - if (!minerCache.hasOwnProperty('goodShares')) { - minerCache.goodShares = 0; - minerCache.badShares = 0; - } - if (!intCache.hasOwnProperty(minerData[0])) { - intCache[minerData[0]] = {paid: 0, due: 0, address: minerData[0], workers: []}; - } - if (typeof(minerData[1]) !== 'undefined') { - intCache[minerData[0]].workers.push({ - worker: minerData[1], - hashRate: minerCache.hash, - lastHash: minerCache.lastHash, - totalHashes: minerCache.totalHashes, - goodShares: minerCache.goodShares, - badShares: minerCache.badShares - }); - } else { - intCache[minerData[0]].lastHash = minerCache.lastHash; - intCache[minerData[0]].totalHashes = minerCache.totalHashes; - intCache[minerData[0]].hashRate = minerCache.hash; - intCache[minerData[0]].goodShares = minerCache.goodShares; - intCache[minerData[0]].badShares = minerCache.badShares; - } - }); - let retList = []; - for (let minerId in intCache) { - if (intCache.hasOwnProperty(minerId)) { - let miner = intCache[minerId]; - retList.push(miner); - } - } - return res.json(retList); - } - return res.json([]); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return res.json({}); }); }); // apply the routes to our application with the prefix /api app.use('/authed', secureRoutes); -app.use('/admin', adminRoutes); // Authenticated routes diff --git a/lib/blockManager.js b/lib/blockManager.js index 89fccc36..4c724dcf 100644 --- a/lib/blockManager.js +++ b/lib/blockManager.js @@ -2,65 +2,50 @@ const range = require("range"); const debug = require("debug")("blockManager"); const async = require("async"); +const fs = require('fs'); +const child_process = require('child_process'); // This file is for managing the block databases within the SQL database. // Primary Tasks: -// Sync the chain into the block_log database. - Scan on startup for missing data, starting from block 0 // Maintain a check for valid blocks in the system. (Only last number of blocks required for validation of payouts) - Perform every 2 minutes. Scan on the main blocks table as well for sanity sake. // Maintain the block_log database in order to ensure payments happen smoothly. - Scan every 1 second for a change in lastblockheader, if it changes, insert into the DB. -let blockIDCache = []; -let scanInProgress = false; -let blockHexCache = {}; -let lastBlock = 0; +let paymentInProgress = false; let balanceIDCache = {}; -let blockScannerTask; -let blockQueue = async.queue(function (task, callback) { - // Todo: Implement within the coins/.js file. - global.support.rpcDaemon('getblockheaderbyheight', {"height": task.blockID}, function (body) { - let blockData = body.result.block_header; - if (blockData.hash in blockHexCache) { - return callback(); + +let createBlockBalanceQueue = async.queue(function (task, callback) { + const sqlq = "REPLACE INTO block_balance (hex, payment_address, payment_id, amount) VALUES ?"; + let sqlp = []; + task.hexes.forEach(function(block_hex) { + sqlp.push([block_hex, task.payment_address, task.payment_id, task.amount]); + }); + global.mysql.query(sqlq, [sqlp]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows < task.hexes.length) { + console.error(JSON.stringify(result)); + console.error("Can't do SQL block balance replace: " + sqlq + " with " + JSON.stringify(sqlp)); + return callback(false); } - debug("Adding block to block_log, ID: " + task.blockID); - blockIDCache.push(task.blockID); - blockHexCache[body.result.block_header.hash] = null; - global.mysql.query("INSERT INTO block_log (id, orphan, hex, find_time, reward, difficulty, major_version, minor_version) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [task.blockID, blockData.orphan_status, blockData.hash, global.support.formatDate(blockData.timestamp * 1000), blockData.reward, blockData.difficulty, blockData.major_version, blockData.minor_version]).then(function () { - return calculatePPSPayments(blockData, callback); - }).catch(function (err) { - debug("BlockHexCache Check: " + blockData.hash in blockHexCache); - debug("BlockIDCache Check: " + blockIDCache.hasOwnProperty(task.blockID)); - debug("Hex: " + blockData.hash + " Height:" + task.blockID); - console.error("Tried to reprocess a block that'd already been processed"); - console.error(JSON.stringify(err)); - return callback(); - }); + return callback(true); + }).catch(function (err) { + console.error(err); + console.error("Can't do SQL block balance replace: " + sqlq + " with " + JSON.stringify(sqlp)); + return callback(false); }); -}, 16); - -blockQueue.drain = function () { - console.log("Scan complete, unlocking remainder of blockManager functionality."); - scanInProgress = false; - if (typeof(blockScannerTask) === 'undefined'){ - blockScannerTask = setInterval(blockScanner, 1000); - } -}; +}, 1); let createBalanceQueue = async.queue(function (task, callback) { let pool_type = task.pool_type; let payment_address = task.payment_address; let payment_id = task.payment_id; - let bitcoin = task.bitcoin; - let query = "SELECT id FROM balance WHERE payment_address = ? AND payment_id is ? AND pool_type = ? AND bitcoin = ?"; + let query = "SELECT id FROM balance WHERE payment_address = ? AND payment_id is ? AND pool_type = ?"; if (payment_id !== null) { - query = "SELECT id FROM balance WHERE payment_address = ? AND payment_id = ? AND pool_type = ? AND bitcoin = ?"; + query = "SELECT id FROM balance WHERE payment_address = ? AND payment_id = ? AND pool_type = ?"; } - let cacheKey = payment_address + pool_type + bitcoin + payment_id; + let cacheKey = payment_address + pool_type + payment_id; debug("Processing a account add/check for:" + JSON.stringify(task)); - global.mysql.query(query, [payment_address, payment_id, pool_type, bitcoin]).then(function (rows) { + global.mysql.query(query, [payment_address, payment_id, pool_type]).then(function (rows) { if (rows.length === 0) { - global.mysql.query("INSERT INTO balance (payment_address, payment_id, pool_type, bitcoin) VALUES (?, ?, ?, ?)", [payment_address, payment_id, pool_type, bitcoin]).then(function (result) { + global.mysql.query("INSERT INTO balance (payment_address, payment_id, pool_type) VALUES (?, ?, ?)", [payment_address, payment_id, pool_type]).then(function (result) { debug("Added to the SQL database: " + result.insertId); balanceIDCache[cacheKey] = result.insertId; return callback(); @@ -74,163 +59,148 @@ let createBalanceQueue = async.queue(function (task, callback) { }, 1); let balanceQueue = async.queue(function (task, callback) { - let pool_type = task.pool_type; - let payment_address = task.payment_address; - let payment_id = null; - if (typeof(task.payment_id) !== 'undefined' && task.payment_id !== null && task.payment_id.length > 10){ - payment_id = task.payment_id; - } - task.payment_id = payment_id; - let bitcoin = task.bitcoin; - let amount = task.amount; - debug("Processing balance increment task: " + JSON.stringify(task)); - async.waterfall([ - function (intCallback) { - let cacheKey = payment_address + pool_type + bitcoin + payment_id; - if (cacheKey in balanceIDCache) { - return intCallback(null, balanceIDCache[cacheKey]); - } else { + const pool_type = task.pool_type; + const amount = task.amount; + const payment_address = task.payment_address; + let payment_id = null; + if (typeof(task.payment_id) !== 'undefined' && task.payment_id !== null && task.payment_id.length > 10) payment_id = task.payment_id; + task.payment_id = payment_id; + debug("Processing balance increment task: " + JSON.stringify(task)); + async.waterfall([ + function (intCallback) { + let cacheKey = payment_address + pool_type + payment_id; + if (cacheKey in balanceIDCache) { + return intCallback(null, balanceIDCache[cacheKey]); + } else { + createBalanceQueue.push(task, function () {}); + async.until(function (untilCB) { + return untilCB(null, cacheKey in balanceIDCache); + }, function (intCallback) { createBalanceQueue.push(task, function () { + return intCallback(null, balanceIDCache[cacheKey]); }); - async.until(function () { - return cacheKey in balanceIDCache; - }, function (intCallback) { - createBalanceQueue.push(task, function () { - return intCallback(null, balanceIDCache[cacheKey]); - }); - }, function () { - return intCallback(null, balanceIDCache[cacheKey]); - } - ); + }, function () { + return intCallback(null, balanceIDCache[cacheKey]); } - }, - function (balance_id, intCallback) { - debug("Made it to the point that I can update the balance for: " + balance_id + " for the amount: " + amount); - global.mysql.query("UPDATE balance SET amount = amount+? WHERE id = ?", [amount, balance_id]).then(function () { - return intCallback(null); - }); - } - ], - function () { - return callback(); - } - ) - ; - }, 24 -); - -function calculatePPSPayments(blockHeader, callback) { - console.log("Performing PPS payout on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward)); - let paymentData = {}; - paymentData[global.config.payout.feeAddress] = { - pool_type: 'fees', - payment_address: global.config.payout.feeAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.coinDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.coinDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.poolDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.poolDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - let totalPayments = 0; - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - for (let found = (cursor.goToRange(blockHeader.height) === blockHeader.height); found; found = cursor.goToNextDup()) { - cursor.getCurrentBinary(function (key, data) { // jshint ignore:line - let shareData; - try { - shareData = global.protos.Share.decode(data); - } catch (e) { - console.error(e); - return; + ); } - let blockDiff = blockHeader.difficulty; - let rewardTotal = blockHeader.reward; - if (shareData.poolType === global.protos.POOLTYPE.PPS) { - let userIdentifier = shareData.paymentAddress; - if (shareData.paymentID) { - userIdentifier = userIdentifier + "." + shareData.paymentID; - } - if (!(userIdentifier in paymentData)) { - paymentData[userIdentifier] = { - pool_type: 'pps', - payment_address: shareData.paymentAddress, - payment_id: shareData.paymentID, - bitcoin: shareData.bitcoin, - amount: 0 - }; - } - let amountToPay = Math.floor((shareData.shares / blockDiff) * rewardTotal); - let feesToPay = Math.floor(amountToPay * (global.config.payout.ppsFee / 100)); - if (shareData.bitcoin === true) { - feesToPay += Math.floor(amountToPay * (global.config.payout.btcFee / 100)); - } - amountToPay -= feesToPay; - paymentData[userIdentifier].amount = paymentData[userIdentifier].amount + amountToPay; - let donations = 0; - if(global.config.payout.devDonation > 0){ - let devDonation = (feesToPay * (global.config.payout.devDonation / 100)); - donations += devDonation; - paymentData[global.coinFuncs.coinDevAddress].amount = paymentData[global.coinFuncs.coinDevAddress].amount + devDonation ; + }, + function (balance_id, intCallback) { + debug("Made it to the point that I can update the balance for: " + balance_id + " for the amount: " + amount); + global.mysql.query("UPDATE balance SET amount = amount+? WHERE id = ?", [amount, balance_id]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do SQL balance update: UPDATE balance SET amount = amount+" + amount + " WHERE id = " + balance_id + ";") } - if(global.config.payout.poolDevDonation > 0){ - let poolDevDonation = (feesToPay * (global.config.payout.poolDevDonation / 100)); - donations += poolDevDonation; - paymentData[global.coinFuncs.poolDevAddress].amount = paymentData[global.coinFuncs.poolDevAddress].amount + poolDevDonation; - } - paymentData[global.config.payout.feeAddress].amount = paymentData[global.config.payout.feeAddress].amount + feesToPay - donations; - } - }); - } - cursor.close(); - txn.abort(); - Object.keys(paymentData).forEach(function (key) { - balanceQueue.push(paymentData[key], function () { - }); - totalPayments += paymentData[key].amount; + return intCallback(null); + }).catch(function (err) { + console.error(err); + console.error("Can't do SQL balance update: UPDATE balance SET amount = amount+" + amount + " WHERE id = " + balance_id + ";") + }); + } + ], + function () { + return callback(); }); - console.log("PPS payout cycle complete on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward) + " Block Payouts: " + global.support.coinToDecimal(totalPayments) + " Payout Percentage: " + (totalPayments / blockHeader.reward) * 100 + "%"); - return callback(); +}, 24); + +let is_full_stop = false; + +function full_stop(err) { + is_full_stop = true; + console.error("Issue making balance increases: " + JSON.stringify(err)); + console.error("Will not make more balance increases until it is resolved!"); + //toAddress, subject, body + global.support.sendEmail(global.config.general.adminEmail, "blockManager unable to make balance increase", + "Hello,\r\nThe blockManager has hit an issue making a balance increase: " + JSON.stringify(err) + + ". Please investigate and restart blockManager as appropriate"); } -function calculatePPLNSPayments(blockHeader) { - console.log("Performing PPLNS payout on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward)); - let rewardTotal = blockHeader.reward; - let blockCheckHeight = blockHeader.height; - let totalPaid = 0; - let paymentData = {}; +let block_unlock_callback = null; +let prev_balance_sum = null; + +balanceQueue.drain(function () { + if (!paymentInProgress) { + debug("balanceQueue.drain: paymentInProgress is false"); + return; + } + if (block_unlock_callback === null) { + debug("balanceQueue.drain: block_unlock_callback is not defined"); + return; + } + console.log("balanceQueue drained: performing block unlocking"); + global.mysql.query("SELECT SUM(amount) as amt FROM balance").then(function (rows) { + if (typeof(rows[0]) === 'undefined' || typeof(rows[0].amt) === 'undefined') { + full_stop("SELECT SUM(amount) as amt FROM balance query returned undefined result"); + block_unlock_callback = null; + prev_balance_sum = null; + paymentInProgress = false; + return; + } + let balance_sum = rows[0].amt; + if (balance_sum !== prev_balance_sum) { + console.log("Total balance changed from " + global.support.coinToDecimal(prev_balance_sum) + " to " + global.support.coinToDecimal(balance_sum) + "\n"); + block_unlock_callback(); + } else { + full_stop("Total balance not changed from " + prev_balance_sum + " to " + balance_sum); + } + block_unlock_callback = null; + prev_balance_sum = null; + paymentInProgress = false; + }); +}); + +function preCalculatePPLNSPayments(block_hexes, block_height, block_difficulty, is_store_dump, done_callback) { + const rewardTotal = 1.0; + console.log("Performing PPLNS reward pre-calculations of block " + block_hexes.join(', ') + " on (anchor) height " + block_height); + const blockDiff = block_difficulty; + const windowPPLNS = blockDiff * global.config.pplns.shareMulti; + + let blockCheckHeight = block_height; + let totalPaid = 0; + let totalShares = 0; + let paymentData = {}; + paymentData[global.config.payout.feeAddress] = { - pool_type: 'fees', + pool_type: 'fees', payment_address: global.config.payout.feeAddress, - payment_id: null, - bitcoin: 0, - amount: 0 + payment_id: null, + amount: 0 }; paymentData[global.coinFuncs.coinDevAddress] = { - pool_type: 'fees', + pool_type: 'fees', payment_address: global.coinFuncs.coinDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 + payment_id: null, + amount: 0 }; paymentData[global.coinFuncs.poolDevAddress] = { - pool_type: 'fees', + pool_type: 'fees', payment_address: global.coinFuncs.poolDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 + payment_id: null, + amount: 0 }; + + function addPayment(keyAdd, valueAdd) { + if (valueAdd === 0) return; + if (totalPaid >= rewardTotal) return; + totalShares += valueAdd; + paymentData[keyAdd].amount += valueAdd; + const totalPaid2 = totalShares / windowPPLNS * rewardTotal; + if (totalPaid2 > rewardTotal) { // totalPaid can not overflow rewardTotal now + //console.log("Value totalPaid " + totalPaid + " reached max " + rewardTotal); + const extra = (totalPaid2 - rewardTotal) / rewardTotal * windowPPLNS; + //console.log("Rewarded " + (valueAdd - extra) + " instead of " + valueAdd + " hashes for " + keyAdd); + paymentData[keyAdd].amount -= extra; + totalPaid = rewardTotal; + } else { + totalPaid = totalPaid2; + } + }; + + let portShares = {}; + let firstShareTime; + let lastShareTime; + let shares4dump = []; + async.doWhilst(function (callback) { let txn = global.database.env.beginTxn({readOnly: true}); let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); @@ -243,270 +213,409 @@ function calculatePPLNSPayments(blockHeader) { console.error(e); return; } - let blockDiff = blockHeader.difficulty; - let rewardTotal = blockHeader.reward; if (shareData.poolType === global.protos.POOLTYPE.PPLNS) { - let userIdentifier = shareData.paymentAddress; - if (shareData.paymentID) { - userIdentifier = userIdentifier + "." + shareData.paymentID; - } + const userIdentifier = shareData.paymentID ? shareData.paymentAddress + "." + shareData.paymentID : shareData.paymentAddress; if (!(userIdentifier in paymentData)) { paymentData[userIdentifier] = { - pool_type: 'pplns', + pool_type: 'pplns', payment_address: shareData.paymentAddress, - payment_id: shareData.paymentID, - bitcoin: shareData.bitcoin, - amount: 0 + payment_id: shareData.paymentID, + amount: 0 }; } - let amountToPay = Math.floor((shareData.shares / (blockDiff*global.config.pplns.shareMulti)) * rewardTotal); - if (totalPaid + amountToPay > rewardTotal) { - amountToPay = rewardTotal - totalPaid; - } - totalPaid += amountToPay; - let feesToPay = Math.floor(amountToPay * (global.config.payout.pplnsFee / 100)); - if (shareData.bitcoin === true) { - feesToPay += Math.floor(amountToPay * (global.config.payout.btcFee / 100)); - } - amountToPay -= feesToPay; - paymentData[userIdentifier].amount = paymentData[userIdentifier].amount + amountToPay; - let donations = 0; - if(global.config.payout.devDonation > 0){ - let devDonation = Math.floor(feesToPay * (global.config.payout.devDonation / 100)); - donations += devDonation; - paymentData[global.coinFuncs.coinDevAddress].amount = paymentData[global.coinFuncs.coinDevAddress].amount + devDonation ; - } - if(global.config.payout.poolDevDonation > 0){ - let poolDevDonation = Math.floor(feesToPay * (global.config.payout.poolDevDonation / 100)); - donations += poolDevDonation; - paymentData[global.coinFuncs.poolDevAddress].amount = paymentData[global.coinFuncs.poolDevAddress].amount + poolDevDonation; + + if (!firstShareTime) firstShareTime = shareData.timestamp; + if (totalPaid < rewardTotal) lastShareTime = shareData.timestamp; + + const amountToPay = shareData.shares2; + const feesToPay = amountToPay * (global.config.payout.pplnsFee / 100); + const devDonation = feesToPay * (global.config.payout.devDonation / 100); + const poolDevDonation = feesToPay * (global.config.payout.poolDevDonation / 100); + const amountToPay2 = amountToPay - feesToPay; + + shares4dump.push(userIdentifier.slice(-16) + "\t" + shareData.timestamp.toString(16) + "\t" + shareData.raw_shares + "\t" + shareData.share_num + "\t" + + global.coinFuncs.PORT2COIN_FULL(shareData.port) + "\t" + amountToPay + "\t" + (amountToPay === amountToPay2 ? "" : amountToPay2)); + + addPayment(userIdentifier, amountToPay2); + addPayment(global.config.payout.feeAddress, feesToPay - devDonation - poolDevDonation); + addPayment(global.coinFuncs.poolDevAddress, poolDevDonation); + addPayment(global.coinFuncs.coinDevAddress, devDonation); + + + if (typeof(shareData.port) !== 'undefined') { + if (shareData.port in portShares) { + portShares[shareData.port] += amountToPay; + } else { + portShares[shareData.port] = amountToPay; + } } - paymentData[global.config.payout.feeAddress].amount = paymentData[global.config.payout.feeAddress].amount + feesToPay - donations; } }); } cursor.close(); txn.abort(); setImmediate(callback, null, totalPaid); - }, function (totalPayment) { + }, function (totalPayment, whilstCB) { blockCheckHeight = blockCheckHeight - 1; debug("Decrementing the block chain check height to:" + blockCheckHeight); if (totalPayment >= rewardTotal) { debug("Loop 1: Total Payment: " + totalPayment + " Amount Paid: " + rewardTotal + " Amount Total: " + totalPaid); - return false; + return whilstCB(null, false); } else { debug("Loop 2: Total Payment: " + totalPayment + " Amount Paid: " + rewardTotal + " Amount Total: " + totalPaid); - return blockCheckHeight !== 0; + return whilstCB(null, blockCheckHeight !== 0); } }, function (err) { + + let sumAllPorts = 0; + for (let port in portShares) sumAllPorts += portShares[port]; + let pplns_port_shares = {}; + for (let port in portShares) { + const port_share = portShares[port] / sumAllPorts; + pplns_port_shares[port] = port_share; + //console.log("Port " + port + ": " + (100.0 * port_share).toFixed(2) + "%"); + } + global.database.setCache('pplns_port_shares', pplns_port_shares); + global.database.setCache('pplns_window_time', (firstShareTime - lastShareTime) / 1000); + let totalPayments = 0; Object.keys(paymentData).forEach(function (key) { - balanceQueue.push(paymentData[key], function () { - }); totalPayments += paymentData[key].amount; }); - console.log("PPLNS payout cycle complete on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward) + " Block Payouts: " + global.support.coinToDecimal(totalPayments) + " Payout Percentage: " + (totalPayments / blockHeader.reward) * 100 + "%"); + + let is_dump_done = false; + let is_ok = true; + let is_pay_done = false; + + if (totalPayments == 0) { + console.warn("PPLNS payout cycle for " + block_hexes.join(', ') + " block does not have any shares so will be redone using top height"); + global.support.sendEmail(global.config.general.adminEmail, + "FYI: No shares to pay block, so it was corrected by using the top height", + "PPLNS payout cycle for " + block_hexes.join(', ') + " block does not have any shares so will be redone using top height" + ); + global.coinFuncs.getLastBlockHeader(function(err, body){ + if (err !== null) { + console.error("Last block header request failed!"); + return done_callback(false); + } + const topBlockHeight = body.height; + return preCalculatePPLNSPayments(block_hexes, topBlockHeight, block_difficulty, is_store_dump, done_callback); + }); + return; + } else { + if (is_store_dump && fs.existsSync("./block_share_dumps/process.sh")) { + shares4dump.sort(); + shares4dump.unshift("#last_16_chars_of_xmr_address\ttimestamp\traw_share_diff\tshare_count\tshare_coin\txmr_share_diff\txmr_share_diff_paid"); + const fn = "block_share_dumps/" + block_hexes[0] + ".cvs"; + fs.writeFile(fn, shares4dump.join("\n"), function(err) { + if (err) { + console.error("Error saving " + fn + " file"); + is_dump_done = true; + if (is_pay_done) return done_callback(is_ok); + return; + } + let fns = ""; + block_hexes.forEach(function(block_hex) { fns += " block_share_dumps/" + block_hex + ".cvs" }); + child_process.exec("./block_share_dumps/process.sh" + fns, function callback(error, stdout, stderr) { + if (error) console.error("./block_share_dumps/process.sh" + fns + ": returned error exit code: " + error.code + "\n" + stdout + "\n" + stderr); + else console.log("./block_share_dumps/process.sh" + fns + ": complete"); + is_dump_done = true; + if (is_pay_done) return done_callback(is_ok); + }); + }); + } else { + is_dump_done = true; + } + } + + const default_window = blockDiff*global.config.pplns.shareMulti; + const is_need_correction = Math.abs(totalPayments/default_window - 1) > 0.0001; + const pay_window = is_need_correction ? totalPayments : default_window; + + let add_count = 0; + + Object.keys(paymentData).forEach(function (key) { + const payment = paymentData[key]; + if (payment.amount) { + const paymentData2 = { + pool_type: 'pplns', + payment_address: payment.payment_address, + payment_id: payment.payment_id, + amount: payment.amount / pay_window, + hexes: block_hexes, + }; + ++ add_count; + createBlockBalanceQueue.push(paymentData2, function (status) { + if (status === false) is_ok = false; + if (--add_count == 0) { + is_pay_done = true; + if (is_dump_done) return done_callback(is_ok); + } + }); + } + }); + + console.log("PPLNS pre-payout cycle complete on block: " + block_height + " Payout Percentage: " + (totalPayments / pay_window) * 100 + "% (precisely " + totalPayments + " / " + pay_window + ")"); + if (is_need_correction) { + console.warn("(This PPLNS payout cycle complete on block was corrected: " + block_height + " Payout Percentage: " + (totalPayments / default_window) * 100 + "% (precisely " + totalPayments + " / " + default_window + "))"); + global.support.sendEmail(global.config.general.adminEmail, + "Warning: Not enought shares to pay block correctly, so it was corrected by upscaling miner rewards!", + "PPLNS payout cycle complete on block: " + block_height + " Payout Percentage: " + (totalPayments / pay_window) * 100 + "% (precisely " + totalPayments + " / " + pay_window + ")\n" + + "(This PPLNS payout cycle complete on block was corrected: " + block_height + " Payout Percentage: " + (totalPayments / default_window) * 100 + "% (precisely " + totalPayments + " / " + default_window + "))" + ); + } }); } -function calculateSoloPayments(blockHeader) { - console.log("Performing Solo payout on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward)); - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - let paymentData = {}; - paymentData[global.config.payout.feeAddress] = { - pool_type: 'fees', - payment_address: global.config.payout.feeAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.coinDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.coinDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - paymentData[global.coinFuncs.poolDevAddress] = { - pool_type: 'fees', - payment_address: global.coinFuncs.poolDevAddress, - payment_id: null, - bitcoin: 0, - amount: 0 - }; - let totalPayments = 0; - for (let found = (cursor.goToRange(blockHeader.height) === blockHeader.height); found; found = cursor.goToNextDup()) { - cursor.getCurrentBinary(function (key, data) { // jshint ignore:line - let shareData; - try { - shareData = global.protos.Share.decode(data); - } catch (e) { - console.error(e); - return; - } - let rewardTotal = blockHeader.reward; - if (shareData.poolType === global.protos.POOLTYPE.SOLO && shareData.foundBlock === true) { - let userIdentifier = shareData.paymentAddress; - if (shareData.paymentID) { - userIdentifier = userIdentifier + "." + shareData.paymentID; - } - if (!(userIdentifier in paymentData)) { - paymentData[userIdentifier] = { - pool_type: 'solo', - payment_address: shareData.paymentAddress, - payment_id: shareData.paymentID, - bitcoin: shareData.bitcoin, - amount: 0 - }; - } - let feesToPay = Math.floor(rewardTotal * (global.config.payout.soloFee / 100)); - if (shareData.bitcoin === true) { - feesToPay += Math.floor(rewardTotal * (global.config.payout.btcFee / 100)); - } - rewardTotal -= feesToPay; - paymentData[userIdentifier].amount = rewardTotal; - let donations = 0; - if(global.config.payout.devDonation > 0){ - let devDonation = (feesToPay * (global.config.payout.devDonation / 100)); - donations += devDonation; - paymentData[global.coinFuncs.coinDevAddress].amount = paymentData[global.coinFuncs.coinDevAddress].amount + devDonation ; - } - if(global.config.payout.poolDevDonation > 0){ - let poolDevDonation = (feesToPay * (global.config.payout.poolDevDonation / 100)); - donations += poolDevDonation; - paymentData[global.coinFuncs.poolDevAddress].amount = paymentData[global.coinFuncs.poolDevAddress].amount + poolDevDonation; - } - paymentData[global.config.payout.feeAddress].amount = feesToPay - donations; +function doPPLNSPayments(block_hex, block_reward, block_port, block_timestamp, unlock_callback) { + console.log("Performing PPLNS payout of block " + block_hex + " with value " + global.support.coinToDecimal(block_reward)); + global.mysql.query("SELECT SUM(amount) as amt FROM balance").then(function (rows) { + if (typeof(rows[0]) === 'undefined' || typeof(rows[0].amt) === 'undefined') { + console.error("SELECT SUM(amount) as amt FROM balance query returned undefined result"); + return; + } + prev_balance_sum = rows[0].amt; + + global.mysql.query("SELECT payment_address, payment_id, amount FROM block_balance WHERE hex = ?", [block_hex]).then(function (rows) { + if (rows.length) { + global.mysql.query("INSERT INTO paid_blocks (hex, amount, port, found_time) VALUES (?,?,?,?)", [block_hex, block_reward, parseInt(block_port), global.support.formatDate(block_timestamp)]).then(function () { + console.log("Adding total due to " + rows.length + " miners"); + block_unlock_callback = unlock_callback; + rows.forEach(function (row) { + row.amount = Math.floor(row.amount * block_reward); + row.pool_type = "pplns"; + balanceQueue.push(row, function () {}); + }); + }).catch(function (error) { + console.error("Block " + block_hex + " can not be inserted into paid_blocks table"); + }); + } else { + console.error("Block " + block_hex + " has no payments in SQL"); } }); + }); +} + +let payReadyBlockHashCalc = {}; + +function blockUnlocker(blockUnlockerCB) { + if (is_full_stop) { + debug("Dropping all block unlocks"); + return blockUnlockerCB(); + } + if (paymentInProgress) { + console.error("Skipping block unlocker run as there's a payment in progress"); + return blockUnlockerCB(); } - cursor.close(); - txn.abort(); - Object.keys(paymentData).forEach(function (key) { - balanceQueue.push(paymentData[key], function () { + console.log("Running block unlocker"); + let blockList = global.database.getValidLockedBlocks(); + global.coinFuncs.getLastBlockHeader(function(err, body){ + if (err !== null) { + console.error("Last block header request failed!"); + return blockUnlockerCB(); + } + const topBlockHeight = body.height; + async.eachSeries(blockList, function(block, next) { + global.coinFuncs.getBlockHeaderByID(block.height, (err, body) => { + if (err !== null) { + console.error("Can't get block with " + block.height + " height"); + return next(); + } + if (topBlockHeight - block.height <= 5) return next(); + const is_pplns_block = block.poolType == global.protos.POOLTYPE.PPLNS; + if (body.hash !== block.hash) { + global.database.invalidateBlock(block.height); + console.log("Invalidating block " + block.height + " due to being an orphan block"); + return next(); + } else if (is_pplns_block && !(block.hash in payReadyBlockHashCalc) && block.pay_ready !== true) { + payReadyBlockHashCalc[block.hash] = 1; + preCalculatePPLNSPayments( [ block.hash ], block.height, block.difficulty, true, function(status) { + if (status) { + console.log("Completed PPLNS reward pre-calculations of block " + block.hash + " on height " + block.height); + global.database.payReadyBlock(block.hash); + } + return next(); + }); + } else if (topBlockHeight - block.height > global.config.payout.blocksRequired && (!is_pplns_block || block.pay_ready === true)) { + blockPayments(block, function() { return next(); } ); + } else { + return next(); + } + }); + }, function() { + return blockUnlockerCB(); }); - totalPayments += paymentData[key].amount; }); - console.log("Solo payout cycle complete on block: " + blockHeader.height + " Block Value: " + global.support.coinToDecimal(blockHeader.reward) + " Block Payouts: " + global.support.coinToDecimal(totalPayments) + " Payout Percentage: " + (totalPayments / blockHeader.reward) * 100 + "%"); } -function blockUnlocker() { - if (scanInProgress) { - debug("Skipping block unlocker run as there's a scan in progress"); - return; +function altblockUnlocker(altblockUnlockerCB) { + if (is_full_stop) { + debug("Dropping all altblock unlocks"); + return altblockUnlockerCB(); } - debug("Running block unlocker"); - let blockList = global.database.getValidLockedBlocks(); - // Todo: Implement within the coins/.js file. - global.support.rpcDaemon('getlastblockheader', [], function (body) { - let blockHeight = body.result.block_header.height; - blockList.forEach(function (row) { - // Todo: Implement within the coins/.js file. - global.support.rpcDaemon('getblockheaderbyheight', {"height": row.height}, function (body) { - if (body.result.block_header.hash !== row.hash) { - global.database.invalidateBlock(row.height); - global.mysql.query("UPDATE block_log SET orphan = true WHERE hex = ?", [row.hash]); - blockIDCache.splice(blockIDCache.indexOf(body.result.block_header.height)); - console.log("Invalidating block " + body.result.block_header.height + " due to being an orphan block"); + if (paymentInProgress) { + console.error("Skipping altblock unlocker run as there's a payment in progress"); + return altblockUnlockerCB(); + } + let blockList = global.database.getValidLockedAltBlocks(); + console.log("Running altblock unlocker for " + blockList.length + " blocks"); + let blockHeightWait = {}; + global.coinFuncs.getLastBlockHeader(function(err, body){ + if (err !== null) { + console.error("Last block header request failed!"); + return altblockUnlockerCB(); + } + const topBlockHeight = body.height; + let preCalcAnchorBlockHashes = {}; + async.eachSeries(blockList, function(block, next) { + if (topBlockHeight - block.anchor_height <= 60) return next(); + const is_pplns_block = block.poolType == global.protos.POOLTYPE.PPLNS; + if (is_pplns_block && !(block.hash in payReadyBlockHashCalc) && block.pay_ready !== true) { + if (block.value) { + const anchor_height = block.anchor_height - (block.anchor_height % global.config.payout.anchorRound); + if (!(anchor_height in preCalcAnchorBlockHashes)) preCalcAnchorBlockHashes[anchor_height] = []; + preCalcAnchorBlockHashes[anchor_height].push(block.hash); + } else global.support.sendEmail(global.config.general.adminEmail, "FYI: blockManager saw zero value locked block", + "Hello,\r\nThe blockManager saw zero value locked block " + block.hash.toString('hex') + ); + return next(); + } else if (!is_pplns_block || block.pay_ready === true) { + if (block.pay_value !== 0) { + console.log(block.port + ": " + block.hash); + global.coinFuncs.getPortBlockHeaderByHash(block.port, block.hash, (err, body) => { + if ( ( body.topoheight && body.topoheight === -1) || + body.confirmations === -1 || + ( body.error instanceof Object && body.error.message === "The requested hash could not be found." ) + ) { + global.database.invalidateAltBlock(block.id); + console.log("Invalidating altblock from " + block.port + " port for " + block.height + " due to being an orphan block"); + return next(); + } else if (err !== null && block.port != 8545 && block.port != 8645) { + console.error("Can't get altblock of " + block.port + " port with " + block.height + " height"); + global.coinFuncs.getPortBlockHeaderByID(block.port, block.height, (err, body) => { + if (err === null && body.hash !== block.hash) { + global.database.invalidateAltBlock(block.id); + console.log("Invalidating altblock from " + block.port + " port for " + block.height + " due to being an orphan block"); + } + return next(); + }); + } else { + altblockPayments(block, function() { return next(); } ); + } + }); } else { - if (blockHeight - row.height > global.config.payout.blocksRequired) { - blockPayments(row); + if (!(block.port in blockHeightWait)) blockHeightWait[block.port] = []; + blockHeightWait[block.port].push(block.height); + return next(); + } + } else { + return next(); + } + + }, function() { + console.log("Running altblock pre-payment for " + Object.keys(preCalcAnchorBlockHashes).length + " anchor heights"); + let maxPreCount = 10; + async.eachSeries(Object.keys(preCalcAnchorBlockHashes), function(anchor_height, next) { + if (--maxPreCount < 0) return next(); + global.coinFuncs.getBlockHeaderByID(anchor_height, function (anchor_err, anchor_header) { + if (anchor_err === null){ + const block_hexes = preCalcAnchorBlockHashes[anchor_height]; + block_hexes.forEach(function (block_hex) { + payReadyBlockHashCalc[block_hex] = 1; + }); + preCalculatePPLNSPayments(block_hexes, parseInt(anchor_height), anchor_header.difficulty, true, function(status) { + if (status) { + console.log("Completed PPLNS reward pre-calculations on altblock " + block_hexes.join(", ") + " on anchor height " + anchor_height + "\n"); + block_hexes.forEach(function (block_hex) { + global.database.payReadyAltBlock(block_hex); + }); + } + return next(); + }); + } else { + console.error("Can't get correct anchor block header by height " + anchor_height); + return next(); } + }); + }, function() { + for (let port in blockHeightWait) { + console.log("Waiting for altblock with " + port + " port and " + blockHeightWait[port].join(", ") + " height(s) pay value"); } + return altblockUnlockerCB(); }); - }); }); } -function blockPayments(block) { +function blockPayments(block, cb) { + if (paymentInProgress) { + console.error("Skipping payment as there's a payment in progress"); + return cb(); + } switch (block.poolType) { - case global.protos.POOLTYPE.PPS: - // PPS is paid out per share find per block, so this is handled in the main block-find loop. - global.database.unlockBlock(block.hash); - break; case global.protos.POOLTYPE.PPLNS: global.coinFuncs.getBlockHeaderByHash(block.hash, function (err, header) { - if (err === null){ - calculatePPLNSPayments(header); - global.database.unlockBlock(block.hash); - } - }); - break; - case global.protos.POOLTYPE.SOLO: - global.coinFuncs.getBlockHeaderByHash(block.hash, function (err, header) { - if (err === null){ - calculateSoloPayments(header); - global.database.unlockBlock(block.hash); + if (err === null && block.height === header.height && block.value === header.reward && block.difficulty === header.difficulty){ + if (paymentInProgress) { + console.error("Skipping payment as there's a payment in progress"); + return cb(); + } + paymentInProgress = true; + doPPLNSPayments(block.hash, block.value, global.config.daemon.port, block.timestamp, function() { + console.log("Unlocking main block on " + block.height + " height with " + block.hash.toString('hex') + " hash"); + global.database.unlockBlock(block.hash); + return cb(); + }); + } else { + console.error("Can't get correct block header by hash " + block.hash.toString('hex')); + global.support.sendEmail(global.config.general.adminEmail, "blockManager unable to make blockPayments", + "Hello,\r\nThe blockManager has hit an issue making blockPayments with block " + block.hash.toString('hex')); + return cb(); } }); break; + default: - console.log("Unknown payment type. FREAKOUT"); - global.database.unlockBlock(block.hash); - break; + console.error("Unknown payment type. FREAKOUT"); + return cb(); } } -function blockScanner() { - let inc_check = 0; - if (scanInProgress) { - debug("Skipping scan as there's one in progress."); - return; +function altblockPayments(block, cb) { + if (paymentInProgress) { + console.error("Skipping payment as there's a payment in progress"); + return cb(); } - scanInProgress = true; - global.coinFuncs.getLastBlockHeader(function (err, blockHeader) { - if (err === null){ - if (lastBlock === blockHeader.height) { - debug("No new work to be performed, block header matches last block"); - scanInProgress = false; - return; - } - debug("Parsing data for new blocks"); - lastBlock = blockHeader.height; - range.range(0, (blockHeader.height - Math.floor(global.config.payout.blocksRequired/2))).forEach(function (blockID) { - if (!blockIDCache.hasOwnProperty(blockID)) { - inc_check += 1; - blockQueue.push({blockID: blockID}, function (err) { - debug("Completed block scan on " + blockID); - if (err) { - console.error("Error processing " + blockID); - } + switch (block.poolType) { + case global.protos.POOLTYPE.PPLNS: + global.coinFuncs.getPortBlockHeaderByHash(block.port, block.hash, function (err, header) { + if (err === null && block.height === header.height && block.value >= header.reward){ + if (paymentInProgress) { + console.error("Skipping payment as there's a payment in progress"); + return cb(); + } + paymentInProgress = true; + doPPLNSPayments(block.hash, block.pay_value, block.port, block.timestamp, function() { + console.log("Unlocking " + block.port + " port block on " + block.height + " height with " + block.hash.toString('hex') + " hash"); + global.database.unlockAltBlock(block.hash); + return cb(); }); + } else { + console.error("Can't get correct altblock header of " + block.port.toString() + " port by hash " + block.hash.toString('hex')); + return cb(); } }); - if (inc_check === 0) { - debug("No new work to be performed, initial scan complete"); - scanInProgress = false; - blockScannerTask = setInterval(blockScanner, 1000); - } - } else { - console.error(`Upstream error from the block daemon. Resetting scanner due to: ${JSON.stringify(blockHeader)}`); - scanInProgress = false; - blockScannerTask = setInterval(blockScanner, 1000); - } - }); + break; + default: + console.error("Unknown payment type. FREAKOUT"); + return cb(); + } } -function initial_sync() { - console.log("Performing boot-sync"); - global.mysql.query("SELECT id, hex FROM block_log WHERE orphan = 0").then(function (rows) { - let intCount = 0; - rows.forEach(function (row) { - intCount += 1; - blockIDCache.push(row.id); - blockHexCache[row.hex] = null; - }); - }).then(function () { - // Enable block scanning for 1 seconds to update the block log. - blockScanner(); - // Scan every 120 seconds for invalidated blocks - setInterval(blockUnlocker, 120000); - blockUnlocker(); - debug("Blocks loaded from SQL: " + blockIDCache.length); - console.log("Boot-sync from SQL complete. Pending completion of queued jobs to get back to work."); - }); +function blockUnlockerNext() { + altblockUnlocker(function() { + setTimeout(blockUnlocker, 2*60*1000, blockUnlockerNext); + }); } -initial_sync(); +blockUnlocker(blockUnlockerNext); diff --git a/lib/coins/aeon.js b/lib/coins/aeon.js deleted file mode 100644 index 4d0f9346..00000000 --- a/lib/coins/aeon.js +++ /dev/null @@ -1,155 +0,0 @@ -"use strict"; -const bignum = require('bignum'); -const cnUtil = require('cryptonote-util'); -const multiHashing = require('multi-hashing'); -const crypto = require('crypto'); -const debug = require('debug')('coinFuncs'); - -let hexChars = new RegExp("[0-9a-f]+"); - -function Coin(data){ - this.bestExchange = global.config.payout.bestExchange; - this.data = data; - let instanceId = crypto.randomBytes(4); - this.coinDevAddress = "WmsSWgtT1JPg5e3cK41hKXSHVpKW7e47bjgiKmWZkYrhSS5LhRemNyqayaSBtAQ6517eo5PtH9wxHVmM78JDZSUu2W8PqRiNs"; // Developer Address - this.poolDevAddress = "WmtvM6SoYya4qzkoPB4wX7FACWcXyFPWAYzfz7CADECgKyBemAeb3dVb3QomHjRWwGS3VYzMJAnBXfUx5CfGLFZd1U7ssdXTu"; // Snipa Address - - this.blockedAddresses = [ - this.coinDevAddress, - this.poolDevAddress, - ]; - - this.exchangeAddresses = [ - ]; // These are addresses that MUST have a paymentID to perform logins with. - - this.prefix = 178; - - this.supportsAutoExchange = false; - - this.niceHashDiff = 200000; - - this.getBlockHeaderByID = function(blockId, callback){ - global.support.rpcDaemon('getblockheaderbyheight', {"height": blockId}, function (body) { - if (body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - callback(true, body); - } - }); - }; - - this.getBlockHeaderByHash = function(blockHash, callback){ - global.support.rpcDaemon('getblockheaderbyhash', {"hash": blockHash}, function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); - } - }); - }; - - this.getLastBlockHeader = function(callback){ - global.support.rpcDaemon('getlastblockheader', [], function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); - } - }); - }; - - this.getBlockTemplate = function(walletAddress, callback){ - global.support.rpcDaemon('getblocktemplate', { - reserve_size: 17, - wallet_address: walletAddress - }, function(body){ - return callback(body); - }); - }; - - this.baseDiff = function(){ - return bignum('FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF', 16); - }; - - this.validateAddress = function(address){ - // This function should be able to be called from the async library, as we need to BLOCK ever so slightly to verify the address. - address = new Buffer(address); - if (cnUtil.address_decode(address) === this.prefix){ - return true; - } - return cnUtil.address_decode_integrated(address) === this.intPrefix; - }; - - this.convertBlob = function(blobBuffer){ - return cnUtil.convert_blob(blobBuffer); - }; - - this.constructNewBlob = function(blockTemplate, NonceBuffer){ - return cnUtil.construct_block_blob(blockTemplate, NonceBuffer); - }; - - this.getBlockID = function(blockBuffer){ - return cnUtil.get_block_id(blockBuffer); - }; - - this.BlockTemplate = function(template) { - /* - Generating a block template is a simple thing. Ask for a boatload of information, and go from there. - Important things to consider. - The reserved space is 13 bytes long now in the following format: - Assuming that the extraNonce starts at byte 130: - |130-133|134-137|138-141|142-145| - |minerNonce/extraNonce - 4 bytes|instanceId - 4 bytes|clientPoolNonce - 4 bytes|clientNonce - 4 bytes| - This is designed to allow a single block template to be used on up to 4 billion poolSlaves (clientPoolNonce) - Each with 4 billion clients. (clientNonce) - While being unique to this particular pool thread (instanceId) - With up to 4 billion clients (minerNonce/extraNonce) - Overkill? Sure. But that's what we do here. Overkill. - */ - - // Set this.blob equal to the BT blob that we get from upstream. - this.blob = template.blocktemplate_blob; - this.idHash = crypto.createHash('md5').update(template.blocktemplate_blob).digest('hex'); - // Set this.diff equal to the known diff for this block. - this.difficulty = template.difficulty; - // Set this.height equal to the known height for this block. - this.height = template.height; - // Set this.reserveOffset to the byte location of the reserved offset. - this.reserveOffset = template.reserved_offset; - // Set this.buffer to the binary decoded version of the BT blob. - this.buffer = new Buffer(this.blob, 'hex'); - // Copy the Instance ID to the reserve offset + 4 bytes deeper. Copy in 4 bytes. - instanceId.copy(this.buffer, this.reserveOffset + 4, 0, 3); - // Generate a clean, shiny new buffer. - this.previous_hash = new Buffer(32); - // Copy in bytes 7 through 39 to this.previous_hash from the current BT. - this.buffer.copy(this.previous_hash, 0, 7, 39); - // Reset the Nonce. - This is the per-miner/pool nonce - this.extraNonce = 0; - // The clientNonceLocation is the location at which the client pools should set the nonces for each of their clients. - this.clientNonceLocation = this.reserveOffset + 12; - // The clientPoolLocation is for multi-thread/multi-server pools to handle the nonce for each of their tiers. - this.clientPoolLocation = this.reserveOffset + 8; - this.nextBlob = function () { - // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Convert the blob into something hashable. - return global.coinFuncs.convertBlob(this.buffer).toString('hex'); - }; - // Make it so you can get the raw block blob out. - this.nextBlobWithChildNonce = function () { - // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Don't convert the blob to something hashable. You bad. - return this.buffer.toString('hex'); - }; - }; - - this.cryptoNight = multiHashing.cryptonight_light; - -} - -module.exports = Coin; diff --git a/lib/coins/krb.js b/lib/coins/krb.js deleted file mode 100644 index 1494da8f..00000000 --- a/lib/coins/krb.js +++ /dev/null @@ -1,155 +0,0 @@ -"use strict"; -const bignum = require('bignum'); -const cnUtil = require('cryptonote-util'); -const multiHashing = require('multi-hashing'); -const crypto = require('crypto'); -const debug = require('debug')('coinFuncs'); - -let hexChars = new RegExp("[0-9a-f]+"); - -function Coin(data){ - this.bestExchange = global.config.payout.bestExchange; - this.data = data; - let instanceId = crypto.randomBytes(4); - this.coinDevAddress = "Kdev1L9V5ow3cdKNqDpLcFFxZCqu5W2GE9xMKewsB2pUXWxcXvJaUWHcSrHuZw91eYfQFzRtGfTemReSSMN4kE445i6Etb3"; // Developer Address - this.poolDevAddress = "KgseWakG2bMXHGJSsAUfzL1HykCyvD4m8gd9qgcuyZ1ufy8PqUCKRxEfAv3nahfdTrCjZByiWoCiRiohxq4u2rf2RgQ1pcJ"; // Snipa Address - - this.blockedAddresses = [ - this.coinDevAddress, - this.poolDevAddress, - ]; - - this.exchangeAddresses = [ - ]; // These are addresses that MUST have a paymentID to perform logins with. - - this.prefix = 111; - - this.supportsAutoExchange = false; - - this.niceHashDiff = 200000; - - this.getBlockHeaderByID = function(blockId, callback){ - global.support.rpcDaemon('getblockheaderbyheight', {"height": blockId}, function (body) { - if (body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); - } - }); - }; - - this.getBlockHeaderByHash = function(blockHash, callback){ - global.support.rpcDaemon('getblockheaderbyhash', {"hash": blockHash}, function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); - } - }); - }; - - this.getLastBlockHeader = function(callback){ - global.support.rpcDaemon('getlastblockheader', [], function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); - } - }); - }; - - this.getBlockTemplate = function(walletAddress, callback){ - global.support.rpcDaemon('getblocktemplate', { - reserve_size: 17, - wallet_address: walletAddress - }, function(body){ - return callback(body); - }); - }; - - this.baseDiff = function(){ - return bignum('FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF', 16); - }; - - this.validateAddress = function(address){ - // This function should be able to be called from the async library, as we need to BLOCK ever so slightly to verify the address. - address = new Buffer(address); - if (cnUtil.address_decode(address) === this.prefix){ - return true; - } - return cnUtil.address_decode_integrated(address) === this.intPrefix; - }; - - this.convertBlob = function(blobBuffer){ - return cnUtil.convert_blob(blobBuffer); - }; - - this.constructNewBlob = function(blockTemplate, NonceBuffer){ - return cnUtil.construct_block_blob(blockTemplate, NonceBuffer); - }; - - this.getBlockID = function(blockBuffer){ - return cnUtil.get_block_id(blockBuffer); - }; - - this.BlockTemplate = function(template) { - /* - Generating a block template is a simple thing. Ask for a boatload of information, and go from there. - Important things to consider. - The reserved space is 13 bytes long now in the following format: - Assuming that the extraNonce starts at byte 130: - |130-133|134-137|138-141|142-145| - |minerNonce/extraNonce - 4 bytes|instanceId - 4 bytes|clientPoolNonce - 4 bytes|clientNonce - 4 bytes| - This is designed to allow a single block template to be used on up to 4 billion poolSlaves (clientPoolNonce) - Each with 4 billion clients. (clientNonce) - While being unique to this particular pool thread (instanceId) - With up to 4 billion clients (minerNonce/extraNonce) - Overkill? Sure. But that's what we do here. Overkill. - */ - - // Set this.blob equal to the BT blob that we get from upstream. - this.blob = template.blocktemplate_blob; - this.idHash = crypto.createHash('md5').update(template.blocktemplate_blob).digest('hex'); - // Set this.diff equal to the known diff for this block. - this.difficulty = template.difficulty; - // Set this.height equal to the known height for this block. - this.height = template.height; - // Set this.reserveOffset to the byte location of the reserved offset. - this.reserveOffset = template.reserved_offset; - // Set this.buffer to the binary decoded version of the BT blob. - this.buffer = new Buffer(this.blob, 'hex'); - // Copy the Instance ID to the reserve offset + 4 bytes deeper. Copy in 4 bytes. - instanceId.copy(this.buffer, this.reserveOffset + 4, 0, 3); - // Generate a clean, shiny new buffer. - this.previous_hash = new Buffer(32); - // Copy in bytes 7 through 39 to this.previous_hash from the current BT. - this.buffer.copy(this.previous_hash, 0, 7, 39); - // Reset the Nonce. - This is the per-miner/pool nonce - this.extraNonce = 0; - // The clientNonceLocation is the location at which the client pools should set the nonces for each of their clients. - this.clientNonceLocation = this.reserveOffset + 12; - // The clientPoolLocation is for multi-thread/multi-server pools to handle the nonce for each of their tiers. - this.clientPoolLocation = this.reserveOffset + 8; - this.nextBlob = function () { - // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Convert the blob into something hashable. - return global.coinFuncs.convertBlob(this.buffer).toString('hex'); - }; - // Make it so you can get the raw block blob out. - this.nextBlobWithChildNonce = function () { - // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Don't convert the blob to something hashable. You bad. - return this.buffer.toString('hex'); - }; - }; - - this.cryptoNight = multiHashing.cryptonight; - -} - -module.exports = Coin; diff --git a/lib/coins/xmr.js b/lib/coins/xmr.js index 558e0fb0..bcfeb1bc 100644 --- a/lib/coins/xmr.js +++ b/lib/coins/xmr.js @@ -1,18 +1,314 @@ "use strict"; const bignum = require('bignum'); -const cnUtil = require('cryptonote-util'); -const multiHashing = require('multi-hashing'); +const cnUtil = require('cryptoforknote-util'); +const multiHashing = require('cryptonight-hashing'); const crypto = require('crypto'); const debug = require('debug')('coinFuncs'); +const process = require('process'); +const fs = require('fs'); +const net = require('net'); +const async = require('async'); +const child_process = require('child_process'); -let hexChars = new RegExp("[0-9a-f]+"); +const reXMRig = /XMRig(?:-[a-zA-Z]+)?\/(\d+)\.(\d+)\./; // 2.8.0 +const reXMRSTAKRX = /\w+-stak-rx\/(\d+)\.(\d+)\.(\d+)/; // 1.0.1 +const reXMRSTAK = /\w+-stak(?:-[a-zA-Z]+)?\/(\d+)\.(\d+)\.(\d+)/; // 2.5.0 +const reXNP = /xmr-node-proxy\/(\d+)\.(\d+)\.(\d+)/; // 0.3.2 +const reCAST = /cast_xmr\/(\d+)\.(\d+)\.(\d+)/; // 1.5.0 +const reSRB = /SRBMiner Cryptonight AMD GPU miner\/(\d+)\.(\d+)\.(\d+)/; // 1.6.8 +const reSRBMULTI = /SRBMiner-MULTI\/(\d+)\.(\d+)\.(\d+)/; // 0.1.5 + +const pool_nonce_size = 16+1; // 1 extra byte for old XMR and new TRTL daemon bugs +const port2coin = { +// "11181": "AEON", +// "11898": "TRTL", + "12211": "RYO", + "17750": "XHV", + "18081": "", +// "18981": "GRFT", + "11812": "XLA", + "25182": "TUBE", +// "34568": "WOW", + "38081": "MSR", + "48782": "LTHN", + "19734": "SUMO", + "13007": "IRD", +// "13102": "XTA", + "19994": "ARQ", + "33124": "XTNC", + "19281": "XMV", + "19950": "XWP", + "9231" : "XEQ", +// "20206": "DERO", +// "18181": "XMC", + "16000": "CCX", + "8766" : "RVN", +// "8545" : "ETH", + "8645" : "ETC", + "2086" : "BLOC", + "9053" : "ERG", + "9998" : "RTM", + "5110" : "KCN", + "10225" : "BTRM", + "17767" : "ZEPH", + "19001" : "XNA", + "9766" : "CLORE", + "19081" : "SAL", +}; +const port2blob_num = { +// "11181": 7, // AEON +// "11898": 2, // TRTL + "12211": 4, // RYO + "17750": 11, // XHV + "18081": 0, // XMR + "17767": 13, // ZEPH +// "18981": 0, // GRFT + "11812": 14, // XLA + "25182": 10, // TUBE +// "34568": 0, // WOW + "38081": 6, // MSR + "48782": 0, // LTHN + "19734": 0, // SUMO + "13007": 2, // IRD +// "13102": 12, // XTA + "19994": 0, // ARQ + "19281": 8, // XMV + "33124": 9, // XTNC + "19950": 8, // XWP + "9231" : 5, // XEQ +// "18181": 0, // XMC + "16000": 0, // CCX + "20206": 100, // DERO + "8766" : 101, // RVN + "8545" : 102, // ETH + "8645" : 102, // ETC + "2086" : 1, // BLOC + "9053" : 103, // ERG + "9998" : 104, // RTM + "5110" : 105, // KCN + "10225": 104, // BTRM + "19001": 101, // XNA + "9766" : 101, // CLORE + "19081": 15, // SAL +}; + +const port2algo = { +// "11181": "k12", // Aeon +// "11898": "argon2/chukwav2", // TRTL + "12211": "cn/gpu", // RYO + "13007": "cn-pico/trtl", // IRD +// "13102": "c29i", // XTA + "17750": "cn-heavy/xhv", // Haven + "18081": "rx/0", // XMR + "17767": "rx/0", // ZEPH +// "18981": "rx/graft", // Graft + "19281": "c29v", // MoneroV + "19734": "cn/r", // SUMO + "19950": "c29s", // Swap + "19994": "rx/arq", // ArqMa + "11812": "panthera", // Scala + "25182": "c29b", // BitTube + "33124": "c29s", // XtendCash +// "34568": "rx/wow", // Wownero + "38081": "cn/half", // MSR + "48782": "argon2/chukwav2", // Lethean + "9231" : "rx/xeq", // XEQ + "20206": "astrobwt/v2", // DERO +// "18181": "cn/0", // XMC + "16000": "cn/gpu", // CCX + "8766" : "kawpow", // RVN + "8545" : "ethash", // ETH + "8645" : "etchash", // ETC + "2086" : "cn-heavy/xhv", // BLOC + "9053" : "autolykos2", // ERG + "9998" : "ghostrider", // RTM + "5110" : "flex", // KCN + "10225": "ghostrider", // BTRM + "19001": "kawpow", // XNA + "9766": "kawpow", // CLORE + "19081": "rx/0", // SAL +}; + +const mm_nonce_size = cnUtil.get_merged_mining_nonce_size(); +const mm_port_set = { }; + +const fix_daemon_sh = "./fix_daemon.sh"; + +const extra_nonce_template_hex = "02" + (pool_nonce_size + 0x100).toString(16).substr(-2) + "00".repeat(pool_nonce_size); +const extra_nonce_mm_template_hex = "02" + (mm_nonce_size + pool_nonce_size + 0x100).toString(16).substr(-2) + "00".repeat(mm_nonce_size + pool_nonce_size); + +function get_coin2port(port2coin) { + let coin2port = {}; + for (let port in port2coin) coin2port[port2coin[port]] = parseInt(port); + return coin2port; +} +const coin2port = get_coin2port(port2coin); +function get_coins(port2coin) { + let coins = []; + for (let port in port2coin) if (port2coin[port] != "") coins.push(port2coin[port]); + return coins; +} +const ports = Object.keys(port2coin); +const coins = get_coins(port2coin); +function get_mm_child_port_set(mm_port_set) { + let mm_child_port_set = {}; + for (let port in mm_port_set) { + const child_port = mm_port_set[port]; + if (!(child_port in mm_child_port_set)) mm_child_port_set[child_port] = {}; + mm_child_port_set[child_port][port] = 1; + } + return mm_child_port_set; +} +function get_algos() { + let algos = {}; + for (let port in port2algo) algos[port2algo[port]] = 1; + return algos; +} +const all_algos = get_algos(); +const mm_child_port_set = get_mm_child_port_set(mm_port_set); + +let miner_address_verify = {}; // store miner address and number of its shares currently in flight for verification +let shareVerifyQueue = []; +let shareVerifyQueueErrorTime = []; +let shareVerifyQueueErrorCount = []; + +if (global.config.verify_shares_host) global.config.verify_shares_host.forEach(function(verify_shares_host, index) { + shareVerifyQueueErrorTime[index] = 0; + shareVerifyQueueErrorCount[index] = 0; + shareVerifyQueue[index] = async.queue(function (task, queueCB) { + if (task.miner_address in miner_address_verify) -- miner_address_verify[task.miner_address]; + const cb = task.cb; + if (Date.now() - task.time > 1*60*1000) { + cb(null); + return queueCB(); + } + + const jsonInput = task.jsonInput; + + let socket = new net.Socket(); + let is_cb = false; + let return_cb = function(result) { + if (is_cb) return; + is_cb = true; + cb(result); + return queueCB(); + } + let timer = setTimeout(function() { + socket.destroy(); + if (shareVerifyQueueErrorCount[index] > 100) { + const err_str = "Server " + global.config.hostname + " timeouted share verification to " + verify_shares_host; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't verify share", err_str); + } + shareVerifyQueueErrorTime[index] = Date.now(); + ++ shareVerifyQueueErrorCount[index]; + return return_cb(false); + }, 60*1000); + socket.connect(2222, verify_shares_host, function () { + socket.write(JSON.stringify(jsonInput) + "\n"); + }); + + let buff = ""; + socket.on('data', function (buff1) { + buff += buff1; + }); + + socket.on("end", function () { + clearTimeout(timer); + timer = null; + try { + const jsonOutput = JSON.parse(buff.toString()); + if (!("result" in jsonOutput)) return return_cb(false); + shareVerifyQueueErrorCount[index] = 0; + return return_cb(jsonOutput.result); + } catch (e) { + if (shareVerifyQueueErrorCount[index] > 100) { + const err_str = "Server " + global.config.hostname + " got wrong JSON from " + verify_shares_host; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't verify share", err_str); + } + shareVerifyQueueErrorTime[index] = Date.now(); + ++ shareVerifyQueueErrorCount[index]; + return return_cb(false); + } + }); + + socket.on('error', function() { + socket.destroy(); + if (shareVerifyQueueErrorCount[index] > 100) { + const err_str = "Server " + global.config.hostname + " got socket error from " + verify_shares_host; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't verify share", err_str); + } + shareVerifyQueueErrorTime[index] = Date.now(); + ++ shareVerifyQueueErrorCount[index]; + return return_cb(false); + }); + }, 16); + + setInterval(function(queue_obj, index){ + if (queue_obj.length() >= 1000) { + let miner_address = {}; + queue_obj.remove(function(task) { + const d = task.data; + if (!(d.miner_address in miner_address)) miner_address[d.miner_address] = 1; + else ++ miner_address[d.miner_address]; + if (Date.now() - d.time > 1*60*1000) { + d.cb(null); + return true; + } + return false; + }); + console.error(global.database.thread_id + "Share verify queue " + index + " state: " + queue_obj.length() + " items in the queue " + queue_obj.running() + " items being processed"); + Object.keys(miner_address).forEach(function(key) { + const value = miner_address[key]; + if (value > 100) console.error("Too many shares from " + key + ": " + value); + }); + } + }, 30*1000, shareVerifyQueue[index], index); +}); + +const ETH_BASE_REWARD = 2; +const ETH_MULTIPLIER = 1000000000000000000; + +function calcEthReward(block, tx_reciepts) { + let gas_prices = {}; + block.transactions.forEach(function(tx) { + gas_prices[tx.hash] = parseInt(tx.gasPrice); + }); + let fee = 0; + tx_reciepts.forEach(function(tx) { + if (tx.result && tx.result.gasUsed) fee += parseInt(tx.result.gasUsed) * gas_prices[tx.result.transactionHash]; + }); + if (block.baseFeePerGas) fee -= parseInt(block.baseFeePerGas) * parseInt(block.gasUsed); + return (ETH_BASE_REWARD + ETH_BASE_REWARD * (block.uncles.length / 32)) * ETH_MULTIPLIER + fee; +} + +function calcErgReward(height, block_tx) { + let reward = 0; + if (block_tx.length && block_tx[0].outputs.length == 2 && block_tx[0].outputs[1].creationHeight == height) { + reward += block_tx[0].outputs[1].value; + // https://docs.ergoplatform.com/dev/protocol/eip27/ + if (block_tx[0].outputs[1].value >= 15000000000) reward -= 12000000000; + else reward -= 3000000000; + } + if (block_tx.length > 1) { + const last_tx = block_tx[block_tx.length - 1]; + if (last_tx.outputs.length == 1 && last_tx.outputs[0].creationHeight == height) { + reward += last_tx.outputs[0].value; + } + } + return reward; +} function Coin(data){ this.bestExchange = global.config.payout.bestExchange; this.data = data; - let instanceId = crypto.randomBytes(4); - this.coinDevAddress = "44AFFq5kSiGBoZ4NMDwYtN18obc8AemS33DBLWs3H7otXft3XjrpDtQGv7SqSsaBYBb98uNbr2VBBEt7f2wfn3RVGQBEP3A"; // Developer Address - this.poolDevAddress = "44Ldv5GQQhP7K7t3ZBdZjkPA7Kg7dhHwk3ZM3RJqxxrecENSFx27Vq14NAMAd2HBvwEPUVVvydPRLcC69JCZDHLT2X5a4gr"; // Snipa Address + let instanceId = Buffer.alloc(4); + instanceId.writeUInt32LE( (((global.config.pool_id % (1<<10)) << 22) + (process.pid % (1<<22))) >>> 0 ); + console.log("Generated instanceId: " + instanceId.toString('hex')); + this.coinDevAddress = "44AFFq5kSiGBoZ4NMDwYtN18obc8AemS33DBLWs3H7otXft3XjrpDtQGv7SqSsaBYBb98uNbr2VBBEt7f2wfn3RVGQBEP3A"; // Monero Developers Address + this.poolDevAddress = "499fS1Phq64hGeqV8p2AfXbf6Ax7gP6FybcMJq6Wbvg8Hw6xms8tCmdYpPsTLSaTNuLEtW4kF2DDiWCFcw4u7wSvFD8wFWE"; // MoneroOcean Address this.blockedAddresses = [ this.coinDevAddress, @@ -20,150 +316,1017 @@ function Coin(data){ "43SLUTpyTgXCNXsL43uD8FWZ5wLAdX7Ak67BgGp7dxnGhLmrffDTXoeGm2GBRm8JjigN9PTg2gnShQn5gkgE1JGWJr4gsEU", // Wolf0's address "42QWoLF7pdwMcTXDviJvNkWEHJ4TXnMBh2Cx6HNkVAW57E48Zfw6wLwDUYFDYJAqY7PLJUTz9cHWB5C4wUA7UJPu5wPf4sZ", // Wolf0's address "46gq64YYgCk88LxAadXbKLeQtCJtsLSD63NiEc3XHLz8NyPAyobACP161JbgyH2SgTau3aPUsFAYyK2RX4dHQoaN1ats6iT", // Claymore's Fee Address. - "47mr7jYTroxQMwdKoPQuJoc9Vs9S9qCUAL6Ek4qyNFWJdqgBZRn4RYY2QjQfqEMJZVWPscupSgaqmUn1dpdUTC4fQsu3yjN" // Claymore's _other_ fee address. + "47mr7jYTroxQMwdKoPQuJoc9Vs9S9qCUAL6Ek4qyNFWJdqgBZRn4RYY2QjQfqEMJZVWPscupSgaqmUn1dpdUTC4fQsu3yjN" // Claymore's _other_ fee address. ]; this.exchangeAddresses = [ "46yzCCD3Mza9tRj7aqPSaxVbbePtuAeKzf8Ky2eRtcXGcEgCg1iTBio6N4sPmznfgGEUGDoBz5CLxZ2XPTyZu1yoCAG7zt6", // Shapeshift.io "463tWEBn5XZJSxLU6uLQnQ2iY9xuNcDbjLSjkn3XAXHCbLrTTErJrBWYgHJQyrCwkNgYvyV3z8zctJLPCZy24jvb3NiTcTJ", // Bittrex "44TVPcCSHebEQp4LnapPkhb2pondb2Ed7GJJLc6TkKwtSyumUnQ6QzkCCkojZycH2MRfLcujCM7QR1gdnRULRraV4UpB5n4", // Xmr.to - "47sghzufGhJJDQEbScMCwVBimTuq6L5JiRixD8VeGbpjCTA12noXmi4ZyBZLc99e66NtnKff34fHsGRoyZk3ES1s1V4QVcB" // Poloniex + "47sghzufGhJJDQEbScMCwVBimTuq6L5JiRixD8VeGbpjCTA12noXmi4ZyBZLc99e66NtnKff34fHsGRoyZk3ES1s1V4QVcB", // Poloniex + "44tLjmXrQNrWJ5NBsEj2R77ZBEgDa3fEe9GLpSf2FRmhexPvfYDUAB7EXX1Hdb3aMQ9FLqdJ56yaAhiXoRsceGJCRS3Jxkn", // Binance.com + "43c2ykU9i2KZHjV8dWff9HKurYYRkckLueYK96Qh4p1EDoEvdo8mpgNJJpPuods53PM6wNzmj4K2D1V11wvXsy9LMiaYc86", // Changelly.com + "45rTtwU6mHqSEMduDm5EvUEmFNx2Z6gQhGBJGqXAPHGyFm9qRfZFDNgDm3drL6wLTVHfVhbfHpCtwKVvDLbQDMH88jx2N6w", // ? + "4ALcw9nTAStZSshoWVUJakZ6tLwTDhixhQUQNJkCn4t3fG3MMK19WZM44HnQRvjqmz4LkkA8t565v7iBwQXx2r34HNroSAZ", // Cryptopia.co.nz + "4BCeEPhodgPMbPWFN1dPwhWXdRX8q4mhhdZdA1dtSMLTLCEYvAj9QXjXAfF7CugEbmfBhgkqHbdgK9b2wKA6nqRZQCgvCDm", // Bitfinex + "41xeYWWKwtSiHju5AdyF8y5xeptuRY3j5X1XYHuB1g6ke4eRexA1iygjXqrT3anyZ22j7DEE74GkbVcQFyH2nNiC3gJqjM9", // HitBTC 1 + "43Kg3mcpvaDhHpv8C4UWf7Kw2DAexn2NoRMqqM5cpAtuRgkedDZWjBQjXqrT3anyZ22j7DEE74GkbVcQFyH2nNiC3dx22mZ", // HitBTC 2 + "44rouyxW44oMc1yTGXBUsL6qo9AWWeHETFiimWC3TMQEizSqqZZPnw1UXCaJrCtUC9QT25L5MZvkoGKRxZttvbkmFXA3TMG", // BTC-Alpha + "45SLfxvu355SpjjzibLKaChA4NGoTrQAwZmSopAXQa9UXBT63BvreEoYyczTcfXow6eL8VaEG2X6NcTG67XZFTNPLgdR9iM", // some web wallet ]; // These are addresses that MUST have a paymentID to perform logins with. this.prefix = 18; + this.subPrefix = 42; this.intPrefix = 19; if (global.config.general.testnet === true){ this.prefix = 53; + this.subPrefix = 63; this.intPrefix = 54; } this.supportsAutoExchange = true; - this.niceHashDiff = 200000; + this.niceHashDiff = 400000; + + this.getPortBlockHeaderByID = function(port, blockId, callback){ + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/' + blockId, null, function (body) { + if (body) { + return callback(null, body); + } else { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + }); + } else if (port == 8766 || port == 9998 || port == 5110 || port == 10225 || port == 19001 || port == 9766) { + global.support.rpcPortDaemon2(port, '', { method: 'getblockhash', params: [ blockId ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + return global.coinFuncs.getPortAnyBlockHeaderByHash(port, body.result, false, callback); + }); + } else if (port == 8545 || port == 8645) { + const blockId2 = blockId === "latest" ? blockId : "0x" + blockId.toString(16); + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getBlockByNumber', params: [ blockId2, true ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + body.result.height = parseInt(body.result.number); + if (blockId === "latest") return callback(null, body.result); // do not need rewards for the latest block + let batch_getBlockReceipts = []; + body.result.transactions.forEach(function(tx) { + batch_getBlockReceipts.push({ jsonrpc: "2.0", id: 1, method: 'eth_getTransactionReceipt', params: [ tx.hash ] }); + }); + if (!batch_getBlockReceipts.length) { + body.result.reward = calcEthReward(body.result, []); + return callback(null, body.result); + } + global.support.rpcPortDaemon2(port, '', batch_getBlockReceipts, function (body2) { + if (!body2 || !(body2 instanceof Array)) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body2)); + return callback(true, body2); + } + body.result.reward = calcEthReward(body.result, body2); + return callback(null, body.result); + }); + }); + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'blocks/at/' + blockId, null, function (body) { + if (!body || !(body instanceof Array) || body.length != 1) { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + return global.coinFuncs.getPortAnyBlockHeaderByHash(port, body[0], false, callback); + }); + } else { + global.support.rpcPortDaemon(port, 'getblockheaderbyheight', {"height": blockId}, function (body) { + if (body && body.hasOwnProperty('result')) { + if (port == 20206) { // DERO is special here + body.result.block_header.timestamp /= 1000; + body.result.block_header.difficulty *= 18; + } + return callback(null, body.result.block_header); + } else { + console.error("getPortBlockHeaderByID(" + port + ", " + blockId + "): " + JSON.stringify(body)); + return callback(true, body); + } + }); + + } + }; this.getBlockHeaderByID = function(blockId, callback){ - global.support.rpcDaemon('getblockheaderbyheight', {"height": blockId}, function (body) { - if (body.hasOwnProperty('result')){ + return this.getPortBlockHeaderByID(global.config.daemon.port, blockId, callback); + }; + + this.getPortAnyBlockHeaderByHash = function(port, blockHash, is_our_block, callback){ + // TRTL/IRD does not get getblock LTHN / AEON / DERO have composite tx + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/' + blockHash, null, function (body) { + if (typeof(body) === 'undefined') { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + return callback(null, body); + }); + } else if (port == 8766 || port == 9998 || port == 5110 || port == 10225 || port == 19001 || port == 9766) { + global.support.rpcPortDaemon2(port, '', { method: 'getblock', params: [ blockHash, 2 ] }, function (body) { + if (!body || !body.result || !(body.result.tx instanceof Array) || body.result.tx.length < 1) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + if (port == 8766 || port == 19001 || port == 9766) { // XNA, RVN, CLORE + body.result.reward = 0; + for (const vout of body.result.tx[0].vout) { + if (vout.scriptPubKey.addresses && vout.scriptPubKey.addresses[0] === "AePr762UcuQrGoa3TRQpGMX6byRjuXw97A") continue; // ignore CLORE common address + body.result.reward += vout.value; + } + } else { // body.result.reward = 3750; // BTRM, RTM + body.result.reward = 0; + for (const vout of body.result.tx[0].vout) { + if (vout.value > body.result.reward) + body.result.reward = vout.value; + } + } + body.result.reward *= 100000000; + body.result.reward = parseInt(body.result.reward); + if (port == 9998 || port == 5110 || port == 10225) body.result.difficulty *= 0xFFFFFFFF; + return callback(null, body.result); + }); + } else if (port == 8545 || port == 8645) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getBlockByHash', params: [ "0x" + blockHash, true ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + body.result.height = parseInt(body.result.number); + global.coinFuncs.getPortBlockHeaderByID(port, body.result.height, function(err, body_height) { + if (err) return callback(true, body); + if (body.result.hash === body_height.hash) { + let batch_getBlockReceipts = []; + body.result.transactions.forEach(function(tx) { + batch_getBlockReceipts.push({ jsonrpc: "2.0", id: 1, method: 'eth_getTransactionReceipt', params: [ tx.hash ] }); + }); + if (!batch_getBlockReceipts.length) { + body.result.reward = calcEthReward(body.result, []); + return callback(null, body.result); + } + global.support.rpcPortDaemon2(port, '', batch_getBlockReceipts, function (body2) { + if (!body2 || !(body2 instanceof Array)) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body2)); + return callback(true, body2); + } + body.result.reward = calcEthReward(body.result, body2); + return callback(null, body.result); + }); + + // uncle block? + } else async.eachSeries(Array(16).fill().map((element, index) => body.result.height + index - 7), function(block_height, next) { + global.coinFuncs.getPortBlockHeaderByID(port, block_height, function(err, body_height) { + if (err) { + if (is_our_block) return next(false); // need to wait for more blocks before it will be reported as uncle + return next(null); + } + const uncleIndex = body_height.uncles.indexOf("0x" + blockHash); + if (uncleIndex === -1) return next(null); + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getUncleByBlockNumberAndIndex', params: [ "0x" + block_height.toString(16), "0x" + uncleIndex.toString(16) ] }, function (body_uncle) { + if (!body_uncle || !body_uncle.result) { + console.error("eth_getUncleByBlockNumberAndIndex(0x" + block_height.toString(16) + ", 0x" + uncleIndex.toString(16) + "): " + JSON.stringify(body_uncle)); + return next(null); + } + return next((ETH_BASE_REWARD * (8 - (parseInt(body_height.number) - parseInt(body_uncle.result.number))) / 8) * ETH_MULTIPLIER); + }); + }); + }, function(uncleReward) { + if (uncleReward === false) return callback(true, body); + body.result.reward = uncleReward ? uncleReward : null; + return callback(null, body.result); + }); + }); + }); + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'blocks/' + blockHash, null, function (body) { + if (!body || !body.header) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } + body.header.reward = calcErgReward(body.header.height, body.blockTransactions.transactions); + return callback(null, body.header); + }); + } else if (port == 13007 || port == 2086 || port == 48782 || port == 11181 || port == 20206 || port == 16000) { + global.support.rpcPortDaemon(port, 'getblockheaderbyhash', {"hash": blockHash}, function (body) { + if ( typeof(body) === 'undefined' || !body.hasOwnProperty('result') ) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); + return callback(true, body); + } return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); + }); + } else global.support.rpcPortDaemon(port, 'getblock', {"hash": blockHash}, function (body) { + if (typeof(body) === 'undefined' || !body.hasOwnProperty('result')) { + console.error("getPortBlockHeaderByHash(" + port + ", " + blockHash + "): " + JSON.stringify(body)); return callback(true, body); } - }); - }; - this.getBlockHeaderByHash = function(blockHash, callback){ - global.support.rpcDaemon('getblockheaderbyhash', {"hash": blockHash}, function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ - return callback(null, body.result.block_header); + body.result.block_header.reward = 0; + if (port == 20206) { // DERO is special here + body.result.block_header.timestamp /= 1000; + body.result.block_header.difficulty *= 18; + } + + let reward_check = 0; + const blockJson = JSON.parse(body.result.json); + const minerTx = blockJson.miner_tx; + + if (port == 17750 || port == 33124 || port == 25182 || port == 18181) { // XHV / XtendCash / TUBE / Italocoin / XMC has reward as zero transaction + reward_check = minerTx.vout[0].amount; } else { - console.error(JSON.stringify(body)); - return callback(true, body); + for (var i=0; i reward_check) { + reward_check = minerTx.vout[i].amount; + } + } } - }); - }; + const miner_tx_hash = body.result.miner_tx_hash == "" ? body.result.block_header.miner_tx_hash : body.result.miner_tx_hash; - this.getLastBlockHeader = function(callback){ - global.support.rpcDaemon('getlastblockheader', [], function (body) { - if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')){ + if (is_our_block && miner_tx_hash) global.support.rpcPortWalletShort(port + 1, "get_transfer_by_txid", {"txid": miner_tx_hash}, function (body2) { + if (typeof(body2) === 'undefined' || body2.hasOwnProperty('error') || !body2.hasOwnProperty('result') || !body2.result.hasOwnProperty('transfer') || !body2.result.transfer.hasOwnProperty('amount')) { + console.error(port + ": block hash: " + blockHash + ": txid " + miner_tx_hash + ": " + JSON.stringify(body2)); + return callback(true, body.result.block_header); + } + let reward = body2.result.transfer.amount; + if (port == 17750) { + body2.result.transfers.forEach(function(transfer) { if (transfer.asset_type === "XHV") reward = transfer.amount; }); + } + + if (reward !== reward_check) { + if (reward_check < reward) { + console.warn(port + ": block hash: " + blockHash + ": txid " + miner_tx_hash + ": using lesser block reward from block header " + reward_check + " instead of higher from incoming wallet tx " + reward); + reward = reward_check; + } else { + console.warn(port + ": block hash: " + blockHash + ": txid " + miner_tx_hash + ": using lesser block reward from incoming tx " + reward + " instead of higher from block header " + reward_check); + } + } + if (port != 38081 && reward == 0) { // MSR can have uncle block reward here + console.error(port + ": block hash: " + blockHash + ": txid " + miner_tx_hash + ": both block header and incoming wallet tx rewards are zero: " + JSON.stringify(body) + "\n" + JSON.stringify(body2)); + return callback(true, body); + } + + body.result.block_header.reward = reward; return callback(null, body.result.block_header); - } else { - console.error(JSON.stringify(body)); - return callback(true, body); + + }); else { + body.result.block_header.reward = reward_check; + return callback(null, body.result.block_header); + } + }); + }; + + this.getPortBlockHeaderByHash = function(port, blockHash, callback){ + return this.getPortAnyBlockHeaderByHash(port, blockHash, true, callback); + }; + + this.getBlockHeaderByHash = function(blockHash, callback){ + return this.getPortBlockHeaderByHash(global.config.daemon.port, blockHash, callback); + }; + + this.getPortLastBlockHeader = function(port, callback, no_error_report) { + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/last', null, function (body) { + if (typeof(body) === 'object') { + return callback(null, body); + } else { + if (!no_error_report) console.error("Last block header invalid: " + JSON.stringify(body)); + return callback(true, body); + } + }); + } else if (port == 8766 || port == 9998 || port == 5110 || port == 10225 || port == 19001 || port == 9766) { + global.support.rpcPortDaemon2(port, '', { method: 'getbestblockhash' }, function (body) { + if (!body || !body.result) { + console.error("getPortLastBlockHeader(" + port + "): " + JSON.stringify(body)); + return callback(true, body); + } + if (global.coinFuncs.lastRavenBlockHash === body.result) return callback(null, global.coinFuncs.lastRavenBlock); + global.coinFuncs.getPortAnyBlockHeaderByHash(port, body.result, false, function (err, body2) { + if (err === null) { + global.coinFuncs.lastRavenBlockHash = body.result; + global.coinFuncs.lastRavenBlock = body2; + } + return callback(err, body2); + }); + }); + } else if (port == 8545 || port == 8645) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: "eth_getWork", "params": [] }, function(body) { + if (!body || !body.result || !(body.result instanceof Array)) return callback(true, body); + const bt = cnUtil.EthBlockTemplate(body.result); + return callback(null, { hash: bt.hash, timestamp: Date.now() / 1000, difficulty: bt.difficulty, height: bt.height, seed_hash: bt.seed_hash }); + }); + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'mining/candidate', null, function(body) { + if (!body || !body.pk) return callback(true, body); + const bt = cnUtil.ErgBlockTemplate(body); + return callback(null, { hash: bt.hash, timestamp: Date.now() / 1000, difficulty: bt.difficulty, height: bt.height, hash2: bt.hash2 }); + }); + } else { + global.support.rpcPortDaemon(port, 'getlastblockheader', [], function (body) { + if (typeof(body) !== 'undefined' && body.hasOwnProperty('result')) { + if (port == 20206) { // DERO is special here + body.result.block_header.timestamp /= 1000; + body.result.block_header.difficulty *= 18; + } + return callback(null, body.result.block_header); + } else { + if (!no_error_report) console.error("Last block header invalid: " + JSON.stringify(body)); + return callback(true, body); + } + }); + } + }; + + this.getLastBlockHeader = function(callback) { + return this.getPortLastBlockHeader(global.config.daemon.port, callback); + }; + + this.getPortBlockTemplate = function(port, callback) { + if (port == 11898) { + global.support.rpcPortDaemon2(port, 'block/template', { + address: global.config.pool["address_" + port.toString()], + reserveSize: port in mm_port_set ? mm_nonce_size + pool_nonce_size : pool_nonce_size + }, function(body) { + return callback(body ? body : null); + }); + + } else if (port == 8766 || port == 9998 || port == 5110 || port == 10225 || port == 19001 || port == 9766) { + global.support.rpcPortDaemon2(port, '', { + method: 'getblocktemplate', + params: [{ + capabilities: [ "coinbasetxn", "workid", "coinbase/append" ], + rules: [ "segwit" ] + }] + }, function(body) { + if (body && body.result) switch (parseInt(port)) { + case 5110: + case 9998: + case 10225: + return callback(cnUtil.RtmBlockTemplate(body.result, global.config.pool["address_" + port.toString()])); + case 8766: + case 19001: + case 9766: + return callback(cnUtil.RavenBlockTemplate(body.result, global.config.pool["address_" + port.toString()])); + } else return callback(null); + }); + + } else if (port == 8545 || port == 8645) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: "eth_getWork", "params": [] }, function(body) { + return callback(body && body.result ? cnUtil.EthBlockTemplate(body.result) : null); + }); + + } else if (port == 9053) { + global.support.rpcPortDaemon2(port, 'mining/candidate', null, function(body) { + return callback(body && body.pk ? cnUtil.ErgBlockTemplate(body) : null); + }); + + + } else { + global.support.rpcPortDaemon(port, 'getblocktemplate', { + reserve_size: port in mm_port_set ? mm_nonce_size + pool_nonce_size : pool_nonce_size, + wallet_address: global.config.pool[port == global.config.daemon.port ? "address" : "address_" + port.toString()] + }, function(body){ + if (body && body.result && port == 20206) { // DERO is special here + body.result.timestamp /= 1000; + body.result.difficulty *= 18; + body.result.mbl_difficulty = body.result.blockhashing_blob.charAt(0) == '4' ? body.result.difficulty : body.result.difficulty * 9; + body.result.reserved_offset = 36; + } + return callback(body && body.result ? body.result : null); + }); + } + }; + + this.getBlockTemplate = function(callback){ + return this.getPortBlockTemplate(global.config.daemon.port, callback); + }; + + + this.ethBlockCheck = function(port, miner_hex, nonce_hex, block_height_hex, callback) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getBlockByNumber', params: [ block_height_hex, true ] }, function (body) { + if (!body || !body.result) { + console.error("getPortBlockHeaderByID(" + port + ", " + block_height_hex + "): " + JSON.stringify(body)); + return callback(null, null); } + if (body.result.miner === miner_hex && body.result.nonce == nonce_hex) return callback(body.result.hash); + block_height_hex = body.result.number; + async.eachSeries(Array(body.result.uncles.length).fill().map((element, index) => index), function(index, next) { + global.support.rpcPortDaemon2(port, '', { jsonrpc: "2.0", id: 1, method: 'eth_getUncleByBlockNumberAndIndex', params: [ block_height_hex, "0x" + index.toString(16) ] }, function (body_uncle) { + if (!body_uncle || !body_uncle.result) { + console.error("eth_getUncleByBlockNumberAndIndex(0x" + block_height_hex + ", 0x" + index.toString(16) + "): " + JSON.stringify(body_uncle)); + return next(null); + } + return next(body_uncle.result.miner === miner_hex && body_uncle.result.nonce == nonce_hex ? body_uncle.result.hash : null); + }); + }, function(block_hash) { + const block_height = parseInt(block_height_hex); + return callback(block_hash, block_height); + }); }); }; - this.getBlockTemplate = function(walletAddress, callback){ - global.support.rpcDaemon('getblocktemplate', { - reserve_size: 17, - wallet_address: walletAddress - }, function(body){ - return callback(body); + this.ethBlockFind = function(port, nonce_hex, callback) { + const miner_hex = global.config.pool["address_" + port]; + global.coinFuncs.ethBlockCheck(port, miner_hex, nonce_hex, "latest", function(block_hash, block_height) { + if (block_hash) return callback(block_hash); + if (!block_height) return callback(null); + async.eachSeries(Array(32).fill().map((element, index) => block_height - index - 1), function(block_height, next) { + global.coinFuncs.ethBlockCheck(port, miner_hex, nonce_hex, "0x" + block_height.toString(16), function(block_hash) { + return next(block_hash); + }); + }, function(block_hash) { + return callback(block_hash); + }); }); }; - this.baseDiff = function(){ - return bignum('FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF', 16); + this.baseDiff = cnUtil.baseDiff; + this.baseRavenDiff = cnUtil.baseRavenDiff; + + this.validatePlainAddress = function(address){ + // This function should be able to be called from the async library, as we need to BLOCK ever so slightly to verify the address. + address = Buffer.from(address); + let code = cnUtil.address_decode(address); + return code === this.prefix || code === this.subPrefix; }; this.validateAddress = function(address){ + if (this.validatePlainAddress(address)) return true; // This function should be able to be called from the async library, as we need to BLOCK ever so slightly to verify the address. - address = new Buffer(address); - if (cnUtil.address_decode(address) === this.prefix){ - return true; - } + address = Buffer.from(address); return cnUtil.address_decode_integrated(address) === this.intPrefix; }; - this.convertBlob = function(blobBuffer){ - return cnUtil.convert_blob(blobBuffer); + this.portBlobType = function(port, version) { return port2blob_num[port]; } + + this.blobTypeGrin = function(blob_type_num) { + switch (blob_type_num) { + case 8: + case 9: + case 10: + case 12: return true; + default: return false; + } + } + + this.c29ProofSize = function(blob_type_num) { + switch (blob_type_num) { + case 10: return 40; + case 12: return 48; + default: return 32; + } + } + + this.nonceSize = function(blob_type_num) { + switch (blob_type_num) { + case 7: + case 101: // RVN + case 102: // ETH + case 103: return 8; // ERG + default: return 4; + } + } + + this.blobTypeDero = function(blob_type_num) { return blob_type_num == 100; } + + this.blobTypeRvn = function(blob_type_num) { return blob_type_num == 101; } + + this.blobTypeEth = function(blob_type_num) { return blob_type_num == 102; } + + this.blobTypeErg = function(blob_type_num) { return blob_type_num == 103; } + + this.blobTypeRtm = function(blob_type_num) { return blob_type_num == 104; } + + this.blobTypeKcn = function(blob_type_num) { return blob_type_num == 105; } + + this.convertBlob = function(blobBuffer, port) { + const blob_type_num = this.portBlobType(port, blobBuffer[0]); + if (this.blobTypeDero(blob_type_num)) return blobBuffer; + let blob; + try { + if (this.blobTypeRvn(blob_type_num)) { + blob = cnUtil.convertRavenBlob(blobBuffer); + } else if (this.blobTypeRtm(blob_type_num)) { + blob = cnUtil.convertRtmBlob(blobBuffer); + } else if (this.blobTypeKcn(blob_type_num)) { + blob = cnUtil.convertKcnBlob(blobBuffer); + } else { + blob = cnUtil.convert_blob(blobBuffer, blob_type_num); + } + } catch (e) { + const err_str = "Can't do port " + port + " convert_blob " + blobBuffer.toString('hex') + " with blob type " + blob_type_num + ": " + e; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't convert_blob", err_str); + return null; + } + return blob; + }; + + this.constructNewBlob = function(blockTemplate, params, port) { + const blob_type_num = this.portBlobType(port, blockTemplate[0]); + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + return cnUtil.construct_block_blob(blockTemplate, + bignum(params.nonce, 10).toBuffer({endian: 'little', size: 4}), + blob_type_num, params.pow + ); + } else if (global.coinFuncs.blobTypeDero(blob_type_num)) { + return cnUtil.constructNewDeroBlob(blockTemplate, Buffer.from(params.nonce, 'hex')); + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + return cnUtil.constructNewRavenBlob(blockTemplate, + bignum(params.nonce, 16).toBuffer({endian: 'little', size: 8}), + bignum(params.mixhash, 16).toBuffer({endian: 'little', size: 32}) + ); + } else if (global.coinFuncs.blobTypeRtm(blob_type_num)) { + return cnUtil.constructNewRtmBlob(blockTemplate, Buffer.from(params.nonce, 'hex')); + } else if (global.coinFuncs.blobTypeKcn(blob_type_num)) { + return cnUtil.constructNewKcnBlob(blockTemplate, Buffer.from(params.nonce, 'hex')); + } else { + return cnUtil.construct_block_blob(blockTemplate, Buffer.from(params.nonce, 'hex'), blob_type_num); + } }; - this.constructNewBlob = function(blockTemplate, NonceBuffer){ - return cnUtil.construct_block_blob(blockTemplate, NonceBuffer); + this.constructMMParentBlockBlob = function(parentTemplateBuffer, port, childTemplateBuffer) { + //console.log("MERGED MINING: constructMMParentBlockBlob"); + return cnUtil.construct_mm_parent_block_blob(parentTemplateBuffer, this.portBlobType(port, parentTemplateBuffer[0]), childTemplateBuffer); }; - this.getBlockID = function(blockBuffer){ - return cnUtil.get_block_id(blockBuffer); + this.constructMMChildBlockBlob = function(shareBuffer, port, childTemplateBuffer) { + console.log("MERGED MINING: constructMMChildBlockBlob"); + return cnUtil.construct_mm_child_block_blob(shareBuffer, this.portBlobType(port, shareBuffer[0]), childTemplateBuffer); + }; + + this.getBlockID = function(blockBuffer, port){ + const blob_type_num = this.portBlobType(port, blockBuffer[0]); + if (global.coinFuncs.blobTypeRtm(blob_type_num)) { + return cnUtil.blockHashBuff(cnUtil.convertRtmBlob(blockBuffer)); + } else if (global.coinFuncs.blobTypeKcn(blob_type_num)) { + return cnUtil.blockHashBuff3(cnUtil.convertKcnBlob(blockBuffer)); + } else { + return cnUtil.get_block_id(blockBuffer, blob_type_num); + } }; this.BlockTemplate = function(template) { - /* - Generating a block template is a simple thing. Ask for a boatload of information, and go from there. - Important things to consider. - The reserved space is 13 bytes long now in the following format: - Assuming that the extraNonce starts at byte 130: - |130-133|134-137|138-141|142-145| - |minerNonce/extraNonce - 4 bytes|instanceId - 4 bytes|clientPoolNonce - 4 bytes|clientNonce - 4 bytes| - This is designed to allow a single block template to be used on up to 4 billion poolSlaves (clientPoolNonce) - Each with 4 billion clients. (clientNonce) - While being unique to this particular pool thread (instanceId) - With up to 4 billion clients (minerNonce/extraNonce) - Overkill? Sure. But that's what we do here. Overkill. - */ - - // Set this.blob equal to the BT blob that we get from upstream. - this.blob = template.blocktemplate_blob; - this.idHash = crypto.createHash('md5').update(template.blocktemplate_blob).digest('hex'); - // Set this.diff equal to the known diff for this block. - this.difficulty = template.difficulty; - // Set this.height equal to the known height for this block. - this.height = template.height; - // Set this.reserveOffset to the byte location of the reserved offset. - this.reserveOffset = template.reserved_offset; - // Set this.buffer to the binary decoded version of the BT blob. - this.buffer = new Buffer(this.blob, 'hex'); + // Generating a block template is a simple thing. Ask for a boatload of information, and go from there. + // Important things to consider. + // The reserved space is 16 bytes long now in the following format: + // Assuming that the extraNonce starts at byte 130: + // |130-133|134-137|138-141|142-145| + // |minerNonce/extraNonce - 4 bytes|instanceId - 4 bytes|clientPoolNonce - 4 bytes|clientNonce - 4 bytes| + // This is designed to allow a single block template to be used on up to 4 billion poolSlaves (clientPoolNonce) + // Each with 4 billion clients. (clientNonce) + // While being unique to this particular pool thread (instanceId) + // With up to 4 billion clients (minerNonce/extraNonce) + // Overkill? Sure. But that's what we do here. Overkill. + + // Set these params equal to values we get from upstream (if they are set) + // DERO-HE case, where mbl is miniblock + this.difficulty = template.mbl_difficulty ? template.mbl_difficulty : template.difficulty; + this.height = template.height; + this.bits = template.bits; + this.seed_hash = template.seed_hash; + this.coin = template.coin; + this.port = template.port; + + const port_blob_num = port2blob_num[this.port]; + + if (template.blocktemplate_blob) { + this.blocktemplate_blob = template.blocktemplate_blob; + } else if (template.blob) { + this.blocktemplate_blob = template.blob; + } else { + const isExtraNonceBT = global.coinFuncs.blobTypeEth(port_blob_num) || global.coinFuncs.blobTypeErg(port_blob_num); + if (isExtraNonceBT) { + const hash = template.hash; + this.hash = this.idHash = hash; + this.hash2 = template.hash2; + this.block_version = 0; + this.nextBlobHex = function () { return hash; }; + return; + } else { + console.error("INTERNAL ERROR: No blob in " + this.port + " port block template: " + JSON.stringify(template)); + this.blocktemplate_blob = extra_nonce_mm_template_hex; // to avoid hard crash + } + } + + const is_mm = "child_template" in template; + + if (is_mm) { + this.child_template = template.child_template; + this.child_template_buffer = template.child_template_buffer; + } + + const is_dero = global.coinFuncs.blobTypeDero(port_blob_num); + const blob = is_dero ? template.blockhashing_blob : (is_mm ? template.parent_blocktemplate_blob : this.blocktemplate_blob); + + this.idHash = crypto.createHash('md5').update(blob).digest('hex'); + + // Set this.buffer to the binary decoded version of the BT blob + this.buffer = Buffer.from(blob, 'hex'); + this.block_version = this.buffer[0]; + + if (global.coinFuncs.blobTypeRvn(port_blob_num) || global.coinFuncs.blobTypeRtm(port_blob_num) || global.coinFuncs.blobTypeKcn(port_blob_num)) { + this.reserved_offset = template.reserved_offset; + } else if (is_dero) { // exception for DERO + this.reserved_offset = template.reserved_offset; + } else { + const template_hex = (template.port in mm_port_set && !is_mm) ? extra_nonce_mm_template_hex : extra_nonce_template_hex; + const found_reserved_offset_template = blob.indexOf(template_hex); + + if (found_reserved_offset_template !== -1) { + const found_reserved_offset = (found_reserved_offset_template >> 1) + 2; + if (is_mm) { + this.reserved_offset = found_reserved_offset; + } else { + if (template.reserved_offset) { + // here we are OK with +1 difference because we put extra byte into pool_nonce_size + if (found_reserved_offset != template.reserved_offset && found_reserved_offset + 1 != template.reserved_offset) { + console.error("INTERNAL ERROR: Found reserved offset " + found_reserved_offset + " do not match " + template.reserved_offset + " reported by daemon in " + this.port + " block " + ": " + blob); + } + this.reserved_offset = template.reserved_offset; + } else if (template.reservedOffset) { + // here we are OK with +1 difference because we put extra byte into pool_nonce_size + if (found_reserved_offset != template.reservedOffset && found_reserved_offset + 1 != template.reservedOffset) { + console.error("INTERNAL ERROR: Found reserved offset " + found_reserved_offset + " do not match " + template.reservedOffset + " reported by daemon in " + this.port + " block " + ": " + blob); + } + this.reserved_offset = template.reservedOffset; + } else { + this.reserved_offset = found_reserved_offset; + } + } + } else { + //console.error("INTERNAL ERROR: Can not find reserved offset template '" + template_hex + "' in " + this.port + " block " + ": " + blob); + this.reserved_offset = template.reserved_offset ? template.reserved_offset : template.reservedOffset; + } + } + + if (!this.reserved_offset) { + console.error("INTERNAL ERROR: No reserved offset in " + this.port + " port block template: " + JSON.stringify(template)); + this.reserved_offset = 0; // to avoid hard crash + } + // Copy the Instance ID to the reserve offset + 4 bytes deeper. Copy in 4 bytes. - instanceId.copy(this.buffer, this.reserveOffset + 4, 0, 3); - // Generate a clean, shiny new buffer. - this.previous_hash = new Buffer(32); - // Copy in bytes 7 through 39 to this.previous_hash from the current BT. - this.buffer.copy(this.previous_hash, 0, 7, 39); - // Reset the Nonce. - This is the per-miner/pool nonce + instanceId.copy(this.buffer, this.reserved_offset + 4, 0, 4); + // Reset the Nonce - this is the per-miner/pool nonce this.extraNonce = 0; // The clientNonceLocation is the location at which the client pools should set the nonces for each of their clients. - this.clientNonceLocation = this.reserveOffset + 12; + this.clientNonceLocation = this.reserved_offset + 12; // The clientPoolLocation is for multi-thread/multi-server pools to handle the nonce for each of their tiers. - this.clientPoolLocation = this.reserveOffset + 8; - this.nextBlob = function () { + this.clientPoolLocation = this.reserved_offset + 8; + + this.nextBlobHex = function () { // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Convert the blob into something hashable. - return global.coinFuncs.convertBlob(this.buffer).toString('hex'); + this.buffer.writeUInt32BE(++this.extraNonce, this.reserved_offset); + // Convert the buffer into something hashable. + const blob = global.coinFuncs.convertBlob(this.buffer, this.port); + return blob ? blob.toString('hex') : null; }; - // Make it so you can get the raw block blob out. - this.nextBlobWithChildNonce = function () { + // Make it so you can get the raw block buffer out. + this.nextBlobWithChildNonceHex = function () { // Write a 32 bit integer, big-endian style to the 0 byte of the reserve offset. - this.buffer.writeUInt32BE(++this.extraNonce, this.reserveOffset); - // Don't convert the blob to something hashable. You bad. + this.buffer.writeUInt32BE(++this.extraNonce, this.reserved_offset); + // Don't convert the buffer to something hashable. You bad. return this.buffer.toString('hex'); }; }; - this.cryptoNight = multiHashing.cryptonight; + this.getPORTS = function() { return ports; } + this.getCOINS = function() { return coins; } + this.PORT2COIN = function(port) { return port2coin[port]; } + this.PORT2COIN_FULL = function(port) { const coin = port2coin[port]; return coin == "" ? "XMR" : coin; } + this.COIN2PORT = function(coin) { return coin2port[coin]; } + this.getMM_PORTS = function() { return mm_port_set; } + this.getMM_CHILD_PORTS = function() { return mm_child_port_set; } + + this.getDefaultAlgos = function() { + return [ "rx/0" ]; + } + + this.getDefaultAlgosPerf = function() { + return { "rx/0": 1 }; + } + + this.getPrevAlgosPerf = function() { + return { "cn/r": 1, "cn/half": 1.9, "cn/rwz": 1.3, "cn/zls": 1.3, "cn/double": 0.5 }; + } + + this.convertAlgosToCoinPerf = function(algos_perf) { + let coin_perf = {}; + + if ("rx/0" in algos_perf) coin_perf[""] = coin_perf["ZEPH"] = coin_perf["SAL"] = algos_perf["rx/0"]; + + if ("cn/r" in algos_perf) coin_perf["SUMO"] = algos_perf["cn/r"]; + + if ("cn/half" in algos_perf) coin_perf["MSR"] = algos_perf["cn/half"]; + else if ("cn/fast2" in algos_perf) coin_perf["MSR"] = algos_perf["cn/fast2"]; + + if ("panthera" in algos_perf) coin_perf["XLA"] = algos_perf["panthera"]; + + if ("cn/gpu" in algos_perf) coin_perf["RYO"] = coin_perf["CCX"] = algos_perf["cn/gpu"]; + + if ("rx/xeq" in algos_perf) coin_perf["XEQ"] = algos_perf["rx/xeq"]; + +// if ("rx/wow" in algos_perf) coin_perf["WOW"] = algos_perf["rx/wow"]; + + if ("kawpow" in algos_perf) coin_perf["RVN"] = coin_perf["XNA"] = coin_perf["CLORE"] = algos_perf["kawpow"]; + if ("kawpow4" in algos_perf) coin_perf["XNA"] = coin_perf["CLORE"] = algos_perf["kawpow4"]; + + if ("ghostrider" in algos_perf) coin_perf["RTM"] = coin_perf["BTRM"] = algos_perf["ghostrider"]; + + if ("flex" in algos_perf) coin_perf["KCN"] = algos_perf["flex"]; + + if ("ethash" in algos_perf) coin_perf["ETC"] = algos_perf["ethash"]; + if ("etchash" in algos_perf) coin_perf["ETC"] = algos_perf["etchash"]; + + if ("autolykos2" in algos_perf) coin_perf["ERG"] = algos_perf["autolykos2"]; + +// if ("rx/graft" in algos_perf) coin_perf["GRFT"] = algos_perf["rx/graft"]; + + if ("cn-heavy/xhv" in algos_perf) coin_perf["XHV"] = coin_perf["BLOC"] = algos_perf["cn-heavy/xhv"]; + + //if ("k12" in algos_perf) coin_perf["AEON"] = algos_perf["k12"]; + + if ("cn-pico" in algos_perf) coin_perf["IRD"] = algos_perf["cn-pico"]; + else if ("cn-pico/trtl" in algos_perf) coin_perf["IRD"] = algos_perf["cn-pico/trtl"]; + + if ("rx/arq" in algos_perf) coin_perf["ARQ"] = algos_perf["rx/arq"]; + + if ("c29s" in algos_perf) coin_perf["XTNC"] = coin_perf["XWP"] = algos_perf["c29s"]; + if ("c29v" in algos_perf) coin_perf["XMV"] = algos_perf["c29v"]; + if ("c29b" in algos_perf) coin_perf["TUBE"] = algos_perf["c29b"]; + + //if ("astrobwt/v2" in algos_perf) coin_perf["DERO"] = algos_perf["astrobwt/v2"]; + + //if ("cn/0" in algos_perf) coin_perf["XMC"] = algos_perf["cn/0"]; + + if ("argon2/chukwav2" in algos_perf) coin_perf["TRTL"] = coin_perf["LTHN"] = algos_perf["argon2/chukwav2"]; + else if ("chukwav2" in algos_perf) coin_perf["TRTL"] = coin_perf["LTHN"] = algos_perf["chukwav2"]; + + return coin_perf; + } + + // returns true if algo set reported by miner is for main algo + this.algoMainCheck = function(algos) { + if ("rx/0" in algos) return true; + return false; + } + // returns true if algo set reported by miner is one of previous main algos + this.algoPrevMainCheck = function(algos) { + if ("cn/r" in algos) return true; + return false; + } + // returns true if algo set reported by miner is OK or error string otherwise + this.algoCheck = function(algos) { + if (this.algoMainCheck(algos)) return true; + for (let algo in all_algos) if (algo in algos) return true; + return "algo array must include at least one supported pool algo: [" + Object.keys(algos).join(", ") + "]"; + } + + this.slowHashBuff = function(convertedBlob, blockTemplate, nonce, mixhash) { + switch (blockTemplate.port) { + case 2086: return multiHashing.cryptonight_heavy(convertedBlob, 1); // BLOC + case 8766: return multiHashing.kawpow(convertedBlob, Buffer.from(nonce, 'hex'), Buffer.from(mixhash, 'hex')); // RVN + case 19001: return multiHashing.kawpow(convertedBlob, Buffer.from(nonce, 'hex'), Buffer.from(mixhash, 'hex')); // XNA + case 9766: return multiHashing.kawpow(convertedBlob, Buffer.from(nonce, 'hex'), Buffer.from(mixhash, 'hex')); // CLORE + case 9998: return multiHashing.cryptonight(convertedBlob, 18); // RTM + case 5110: return multiHashing.cryptonight(convertedBlob, 19); // KCN + case 10225: return multiHashing.cryptonight(convertedBlob, 18); // BTRM + case 8545: return multiHashing.ethash(convertedBlob, Buffer.from(nonce, 'hex'), blockTemplate.height); // ETH + case 8645: return multiHashing.etchash(convertedBlob, Buffer.from(nonce, 'hex'), blockTemplate.height); // ETC + case 9053: return multiHashing.autolykos2_hashes(convertedBlob, blockTemplate.height); // ERG + case 9231 : return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 22); // XEQ + //case 11181: return multiHashing.k12(convertedBlob); // Aeon + case 11898: return multiHashing.argon2(convertedBlob, 2); // TRTL + case 12211: return multiHashing.cryptonight(convertedBlob, 11); // RYO + case 13007: return multiHashing.cryptonight_pico(convertedBlob, 0); // Iridium + case 16000: return multiHashing.cryptonight(convertedBlob, 11); // CCX + case 17750: return multiHashing.cryptonight_heavy(convertedBlob, 1); // Haven + case 18081: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 0); // XMR + case 17767: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 0); // ZEPH + case 19081: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 0); // SAL + //case 18181: return multiHashing.cryptonight(convertedBlob, 0); // XMC +// case 18981: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 20); // Graft + case 19734: return multiHashing.cryptonight(convertedBlob, 13, blockTemplate.height); // SUMO + case 19994: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 2); // ArqMa + case 11812: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 3); // Scala + case 20206: return multiHashing.astrobwt(convertedBlob, 1); // Dero +// case 34568: return multiHashing.randomx(convertedBlob, Buffer.from(blockTemplate.seed_hash, 'hex'), 17); // Wownero + case 38081: return multiHashing.cryptonight(convertedBlob, 9); // MSR + case 48782: return multiHashing.argon2(convertedBlob, 2); // Lethean + default: + console.error("Unknown " + blockTemplate.port + " port for Cryptonight PoW type"); + return multiHashing.cryptonight(convertedBlob, 13, blockTemplate.height); + } + } + + this.slowHash = function(convertedBlob, blockTemplate, nonce, mixhash) { + return this.slowHashBuff(convertedBlob, blockTemplate, nonce, mixhash).toString("hex"); + } + + this.verify_share_host_index = 0; + + this.slowHashAsync = function(convertedBlob, blockTemplate, miner_address, cb) { + if (!global.config.verify_shares_host) return cb(this.slowHash(convertedBlob, blockTemplate)); + if (miner_address in miner_address_verify) { + if (miner_address_verify[miner_address] > 100) return cb(null); + ++ miner_address_verify[miner_address]; + } else miner_address_verify[miner_address] = 1; + let jsonInput; + switch (blockTemplate.port) { + case 9231: + case 17767: + case 18081: + case 19081: +// case 18981: + case 19994: + case 11812: + case 22023: +// case 34568: + jsonInput = { "algo": port2algo[blockTemplate.port], "blob": convertedBlob.toString('hex'), "seed_hash": blockTemplate.seed_hash }; + break; + case 19734: + jsonInput = { "algo": port2algo[blockTemplate.port], "blob": convertedBlob.toString('hex'), "height": blockTemplate.height }; + break; + //case 11181: + // return cb(this.slowHash(convertedBlob, blockTemplate)); // AEON K12 is too fast + default: + jsonInput = { "algo": port2algo[blockTemplate.port], "blob": convertedBlob.toString('hex') }; + } + const time_now = Date.now(); + let best_index = null; + let min_queue_size = null; + let max_noerr_time = null; + shareVerifyQueue.forEach(function(queue_obj, index) { + if (time_now - shareVerifyQueueErrorTime[index] < 1*60*1000 && shareVerifyQueueErrorCount[index] > 100 && global.config.verify_shares_host[index] !== "127.0.0.1") return; + const qlength = queue_obj.length() + queue_obj.running(); + if (min_queue_size === null || qlength < min_queue_size) { + best_index = index; + min_queue_size = qlength; + } + }); + if (best_index === null) shareVerifyQueueErrorTime.forEach(function(last_error_time, index) { + const noerr_time = time_now - last_error_time; + if (max_noerr_time === null || noerr_time > max_noerr_time) { + best_index = index; + max_noerr_time = noerr_time; + } + }); + return shareVerifyQueue[best_index].unshift({ + jsonInput: jsonInput, + cb: cb, + time: time_now, + miner_address: miner_address + }); + } + + this.c29 = function(header, ring, port) { + switch (port) { + case 19281: return multiHashing.c29v(header, ring); // MoneroV + case 19950: return multiHashing.c29s(header, ring); // Swap + case 25182: return multiHashing.c29b(header, ring); // TUBE + case 33124: return multiHashing.c29s(header, ring); // XtendCash + default: + console.error("Unknown " + port + " port for Cuckaroo PoW type"); + return multiHashing.c29s(header, ring); + } + } + + this.c29_cycle_hash = function(ring, blob_type_num) { + switch (blob_type_num) { + case 10: return multiHashing.c29b_cycle_hash(ring); + case 12: return multiHashing.c29i_cycle_hash(ring); + default: return multiHashing.c29_cycle_hash(ring); + } + } + + this.blobTypeStr = function(port, version) { + switch (port) { + case 2086: return "forknote1"; // BLOC + case 8545: return "eth"; // ETH + case 8645: return "eth"; // ETC + case 8766: return "raven"; // RVN + case 19001: return "raven"; // XNA + case 9766: return "raven"; // CLORE + case 9053: return "erg"; // ERG + case 9231: return "cryptonote_loki"; // XEQ + case 9998: return "raptoreum"; // RTM + case 5110: return "raptoreum_kcn"; // KCN + case 10225: return "raptoreum"; // BTRM + //case 11181: return "aeon"; // Aeon + case 11898: return "forknote2"; // TRTL + case 13007: return "forknote2"; // Iridium + case 12211: return "cryptonote_ryo"; // RYO + case 17750: return "cryptonote_xhv"; // XHV + case 17767: return "cryptonote_zeph"; // ZEPH + case 19081: return "cryptonote_sal"; // SAL + case 19281: return "cuckaroo"; // MoneroV + case 19950: return "cuckaroo"; // Swap + case 20206: return "cryptonote_dero"; // Dero + case 22023: return "cryptonote_loki"; // LOKI + case 25182: return "cryptonote_tube"; // TUBE + case 33124: return "cryptonote_xtnc"; // XtendCash + case 38081: return "cryptonote3"; // MSR + case 11812: return "cryptonote_xla"; // XLA + default: return "cryptonote"; + } + } + + this.algoShortTypeStr = function(port, version) { + if (port in port2algo) return port2algo[port]; + console.error("Unknown " + port + " port for PoW type on " + version + " version"); + return "rx/0"; + } + + this.isMinerSupportAlgo = function(algo, algos) { + if (algo in algos) return true; + if (algo === "cn-heavy/0" && "cn-heavy" in algos) return true; + return false; + } + + this.get_miner_agent_warning_notification = function(agent) { + let m; + if (m = reXMRig.exec(agent)) { + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + if (majorv + minorv < 30200) { + return "Please update your XMRig miner (" + agent + ") to v3.2.0+ to support new rx/0 Monero algo"; + } + if (majorv + minorv >= 40000 && majorv + minorv < 40200) { + return "Please update your XMRig miner (" + agent + ") to v4.2.0+ to support new rx/0 Monero algo"; + } + } else if (m = reXMRSTAKRX.exec(agent)) { + return false; + } else if (m = reXMRSTAK.exec(agent)) { + return "Please update your xmr-stak miner (" + agent + ") to xmr-stak-rx miner to support new rx/0 Monero algo"; + } else if (m = reXNP.exec(agent)) { + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + const minorv2 = parseInt(m[3]); + const version = majorv + minorv + minorv2; + if (version < 1400) { + return "Please update your xmr-node-proxy (" + agent + ") to version v0.14.0+ by doing 'cd xmr-node-proxy && ./update.sh' (or check https://github.com/MoneroOcean/xmr-node-proxy repo) to support new rx/0 Monero algo"; + } + } else if (m = reSRBMULTI.exec(agent)) { + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + const minorv2 = parseInt(m[3]); + if (majorv + minorv + minorv2 < 105) { + return "Please update your SRBminer-MULTI (" + agent + ") to version v0.1.5+ to support new rx/0 Monero algo"; + } + } + return false; + }; + + this.is_miner_agent_no_haven_support = function(agent) { + let m; + if (m = reXMRig.exec(agent)) { + const majorv = parseInt(m[1]) * 10000; + const minorv = parseInt(m[2]) * 100; + if (majorv + minorv < 60300) { + return true; + } + } + return false; + }; + + this.get_miner_agent_not_supported_algo = function(agent) { + let m; + if (m = reXMRSTAKRX.exec(agent)) { + return "rx/0"; + } else if (m = reXMRSTAK.exec(agent)) { + return "cn/r"; + } + return false; + }; + + this.fixDaemonIssue = function(height, top_height, port) { + global.support.sendEmail(global.config.general.adminEmail, + "Pool server " + global.config.hostname + " has stuck block template", + "The pool server: " + global.config.hostname + " with IP: " + global.config.bind_ip + " with current block height " + + height + " is stuck compared to top height (" + top_height + ") amongst other leaf nodes for " + + port + " port\nAttempting to fix..." + ); + if (fs.existsSync(fix_daemon_sh)) { + child_process.exec(fix_daemon_sh + " " + port, function callback(error, stdout, stderr) { + console.log("> " + fix_daemon_sh + " " + port); + console.log(stdout); + console.error(stderr); + if (error) console.error(fix_daemon_sh + " script returned error exit code: " + error.code); + }); + } else { + console.error("No " + fix_daemon_sh + " script was found to fix stuff"); + } + } +}; + -} module.exports = Coin; diff --git a/lib/data.proto b/lib/data.proto index 0db14213..030c2999 100644 --- a/lib/data.proto +++ b/lib/data.proto @@ -9,6 +9,7 @@ enum MESSAGETYPE { SHARE = 0; BLOCK = 1; INVALIDSHARE = 2; + ALTBLOCK = 3; } message WSData { @@ -22,10 +23,11 @@ message InvalidShare{ required string paymentAddress = 1; optional string paymentID = 2; required string identifier = 3; + optional int32 count = 4; } message Share { - required int32 shares = 1; + optional int64 shares = 1; required string paymentAddress = 2; required bool foundBlock = 3; optional string paymentID = 4; @@ -33,10 +35,13 @@ message Share { required POOLTYPE poolType = 6; required int32 poolID = 7; required int64 blockDiff = 8; - required bool bitcoin = 9; required int32 blockHeight = 10; required int64 timestamp = 11; required string identifier = 12; + optional int32 port = 13; + optional int64 shares2 = 14; + optional int64 share_num = 15; + optional float raw_shares = 16; } message Block { @@ -48,4 +53,23 @@ message Block { required bool unlocked = 6; required bool valid = 7; optional int64 value = 8; -} \ No newline at end of file + optional bool pay_ready = 9; +} + +message AltBlock { + required string hash = 1; + required int64 difficulty = 2; + required int64 shares = 3; + required int64 timestamp = 4; + required POOLTYPE poolType = 5; + required bool unlocked = 6; + required bool valid = 7; + required int32 port = 8; + required int32 height = 9; + required int32 anchor_height = 10; + optional uint64 value = 11; + optional uint64 pay_value = 12; + optional string pay_stage = 13; + optional string pay_status = 14; + optional bool pay_ready = 15; +} diff --git a/lib/local_comms.js b/lib/local_comms.js index b86bea4e..43148f7d 100644 --- a/lib/local_comms.js +++ b/lib/local_comms.js @@ -2,26 +2,36 @@ let range = require('range'); let debug = require('debug')('db'); let async = require('async'); +let cleanShareInProgress = false; +let cleanShareStuckCount = 0; + +function poolTypeStr(poolType) { + switch (poolType) { + case global.protos.POOLTYPE.PPLNS: return 'pplns'; + case global.protos.POOLTYPE.PPS: return 'pps'; + case global.protos.POOLTYPE.SOLO: return 'solo'; + default: + console.error("Unknown poolType: " + poolType.toString()); + return 'pplns'; + } +} function Database(){ this.lmdb = require('node-lmdb'); this.env = null; this.shareDB = null; this.blockDB = null; + this.altblockDB = null; this.cacheDB = null; - this.dirtyenv = false; this.initEnv = function(){ global.database.env = new this.lmdb.Env(); global.database.env.open({ path: global.config.db_storage_path, maxDbs: 10, - mapSize: 24 * 1024 * 1024 * 1024, - noSync: false, - mapAsync: true, - useWritemap: false, - noMetaSync: true, + mapSize: global.config.general.dbSizeGB * 1024 * 1024 * 1024, + useWritemap: true, maxReaders: 512 }); global.database.shareDB = this.env.openDbi({ @@ -39,29 +49,29 @@ function Database(){ integerKey: true, keyIsUint32: true }); + global.database.altblockDB = this.env.openDbi({ + name: 'altblocks', + create: true, + integerKey: true, + keyIsUint32: true + }); global.database.cacheDB = this.env.openDbi({ name: 'cache', create: true }); - global.database.intervalID = setInterval(function(){ - global.database.env.sync(function(){}); - }, 60000); // Sync the DB every 60 seconds - global.database.dirtyenv = false; + //global.database.intervalID = setInterval(function(){ + // global.database.env.sync(function(){}); + //}, 60000); // Sync the DB every 60 seconds console.log("Database Worker: LMDB Env Initialized."); }; - this.incrementCacheData = function(key, data){ - this.refreshEnv(); let txn = this.env.beginTxn(); let cached = txn.getString(this.cacheDB, key); if (cached !== null){ cached = JSON.parse(cached); data.forEach(function(intDict){ - if (!cached.hasOwnProperty(intDict.location)){ - cached[intDict.location] = 0; - } - if (intDict.value === false){ + if (!cached.hasOwnProperty(intDict.location) || intDict.value === false){ cached[intDict.location] = 0; } else { cached[intDict.location] += intDict.value; @@ -69,12 +79,12 @@ function Database(){ }); txn.putString(this.cacheDB, key, JSON.stringify(cached)); txn.commit(); - return; + } else { + txn.abort(); } - txn.abort(); }; - this.getBlockList = function(pool_type){ + this.getBlockList = function(pool_type, first, last) { debug("Getting block list"); switch (pool_type) { case 'pplns': @@ -91,37 +101,15 @@ function Database(){ } let response = []; try{ - this.refreshEnv(); let txn = global.database.env.beginTxn({readOnly: true}); let cursor = new global.database.lmdb.Cursor(txn, global.database.blockDB); - for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { - /* - required string hash = 1; - required int64 difficulty = 2; - required int64 shares = 3; - required int64 timestamp = 4; - required POOLTYPE poolType = 5; - required bool unlocked = 6; - required bool valid = 7; - */ + for (let found = cursor.goToLast(), i = 0; found; found = cursor.goToPrev()) { + if (typeof last !== 'undefined' && i >= last) break; cursor.getCurrentBinary(function (key, data) { // jshint ignore:line let blockData = global.protos.Block.decode(data); - let poolType; - switch (blockData.poolType){ - case (global.protos.POOLTYPE.PPLNS): - poolType = 'pplns'; - break; - case (global.protos.POOLTYPE.SOLO): - poolType = 'solo'; - break; - case (global.protos.POOLTYPE.PPS): - poolType = 'pps'; - break; - default: - poolType = 'Unknown'; - break; - } - if (blockData.poolType === pool_type || pool_type === false) { + let poolType = poolTypeStr(blockData.poolType); + if (pool_type === false || blockData.poolType === pool_type) { + if (typeof first !== 'undefined' && i++ < first) return; response.push({ ts: blockData.timestamp, hash: blockData.hash, @@ -138,61 +126,75 @@ function Database(){ } cursor.close(); txn.abort(); - return response.sort(global.support.blockCompare); + return response; //.sort(global.support.blockCompare); } catch (e){ return response; } }; - this.storeShare = function(blockId, shareData, callback){ - // This function needs the blockID in question, and the shareData in binary format. - // The binary data should be packed as per the data.proto Share protobuf format. - try { - let share = global.protos.Share.decode(shareData); - let minerID = share.paymentAddress; - if (typeof(share.paymentID) !== 'undefined' && share.paymentID.length > 10) { - minerID = minerID + '.' + share.paymentID; - } - let minerIDWithIdentifier = minerID + "_" + share.identifier; - this.incrementCacheData('global_stats', [{location: 'totalHashes', value: share.shares}, {location: 'roundHashes', value: share.shares}]); - switch (share.poolType) { - case global.protos.POOLTYPE.PPLNS: - this.incrementCacheData('pplns_stats', [{location: 'totalHashes', value: share.shares}, {location: 'roundHashes', value: share.shares}]); - break; - case global.protos.POOLTYPE.PPS: - this.incrementCacheData('pps_stats', [{location: 'totalHashes', value: share.shares}, {location: 'roundHashes', value: share.shares}]); - break; - case global.protos.POOLTYPE.SOLO: - this.incrementCacheData('solo_stats', [{location: 'totalHashes', value: share.shares}, {location: 'roundHashes', value: share.shares}]); - break; + this.getAltBlockList = function(pool_type, coin_port, first, last) { + debug("Getting altblock list"); + switch (pool_type) { + case 'pplns': + pool_type = global.protos.POOLTYPE.PPLNS; + break; + case 'pps': + pool_type = global.protos.POOLTYPE.PPS; + break; + case 'solo': + pool_type = global.protos.POOLTYPE.SOLO; + break; + default: + pool_type = false; + } + let response = []; + try{ + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToLast(), i = 0; found; found = cursor.goToPrev()) { + if (typeof last !== 'undefined' && i >= last) break; + cursor.getCurrentBinary(function (key, data) { // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + let poolType = poolTypeStr(blockData.poolType); + if ((pool_type === false || blockData.poolType === pool_type) && (!coin_port || blockData.port === coin_port)) { + if (typeof first !== 'undefined' && i++ < first) return; + response.push({ + ts: blockData.timestamp, + hash: blockData.hash, + diff: blockData.difficulty, + shares: blockData.shares, + height: blockData.height, + valid: blockData.valid, + unlocked: blockData.unlocked, + pool_type: poolType, + value: blockData.value, + pay_value: blockData.pay_value, + pay_stage: blockData.pay_stage, + pay_status: blockData.pay_status, + port: blockData.port + }); + } + }); } - this.incrementCacheData(minerIDWithIdentifier, [{location: 'totalHashes', value: share.shares},{location: 'goodShares', value: 1}]); - this.incrementCacheData(minerID, [{location: 'totalHashes', value: share.shares},{location: 'goodShares', value: 1}]); + cursor.close(); + txn.abort(); + return response; //.sort(global.support.tsCompare); } catch (e){ - callback(false); - return; + return response; } - this.refreshEnv(); - let txn = this.env.beginTxn(); - txn.putBinary(this.shareDB, blockId, shareData); - txn.commit(); - callback(true); }; this.storeBulkShares = function(shareObject) { let cachedData = { - global_stats: {totalHashes: 0, roundHashes: 0}, - pplns_stats: {totalHashes: 0, roundHashes: 0}, - pps_stats: {totalHashes: 0, roundHashes: 0}, - solo_stats: {totalHashes: 0, roundHashes: 0} + global_stats2: { totalHashes: 0, roundHashes: 0 } }; let shares = {}; // Shares keyed by blockID let shareCount = 0; debug(shareObject.length + ' shares to process'); shareObject.forEach(function(share){ //Data is the share object at this point. - shareCount += 1; - if (typeof(share.shares) === "number") { + ++ shareCount; + if (typeof(share.raw_shares) === "number") { if (!shares.hasOwnProperty(share.blockHeight)) { shares[share.blockHeight] = []; } @@ -202,32 +204,40 @@ function Database(){ minerID = minerID + '.' + share.paymentID; } let minerIDWithIdentifier = minerID + "_" + share.identifier; - cachedData.global_stats.totalHashes += share.shares; - cachedData.global_stats.roundHashes += share.shares; - let stats_type = 'pplns_stats'; - switch (share.poolType) { - case global.protos.POOLTYPE.PPLNS: - stats_type = 'pplns_stats'; - break; - case global.protos.POOLTYPE.PPS: - stats_type = 'pps_stats'; - break; - case global.protos.POOLTYPE.SOLO: - stats_type = 'solo_stats'; - break; + let port_suffix = typeof(share.port) !== 'undefined' && share.port !== global.config.daemon.port ? "_" + share.port.toString() : ""; + let global_stats1 = "global_stats2"; + let stats_type1 = poolTypeStr(share.poolType) + "_stats2"; + if (!(stats_type1 in cachedData)) { + cachedData[stats_type1] = { totalHashes: 0, roundHashes: 0 }; + } + if (port_suffix === "") { + cachedData[global_stats1].totalHashes += share.raw_shares; + cachedData[global_stats1].roundHashes += share.raw_shares; + cachedData[stats_type1].totalHashes += share.raw_shares; + cachedData[stats_type1].roundHashes += share.raw_shares; + } else { + let global_stats2 = global_stats1 + port_suffix; + let stats_type2 = stats_type1 + port_suffix; + if (!(global_stats2 in cachedData)) cachedData[global_stats2] = { totalHashes: 0, roundHashes: 0 }; + if (!(stats_type2 in cachedData)) cachedData[stats_type2] = { totalHashes: 0, roundHashes: 0 }; + cachedData[global_stats1].totalHashes += share.raw_shares; + cachedData[global_stats2].totalHashes += share.raw_shares; + cachedData[global_stats2].roundHashes += share.raw_shares; + cachedData[stats_type1].totalHashes += share.raw_shares; + cachedData[stats_type2].totalHashes += share.raw_shares; + cachedData[stats_type2].roundHashes += share.raw_shares; } - cachedData[stats_type].totalHashes += share.shares; - cachedData[stats_type].roundHashes += share.shares; if (!cachedData.hasOwnProperty(minerID)) { cachedData[minerID] = {totalHashes: 0, goodShares: 0}; } if (!cachedData.hasOwnProperty(minerIDWithIdentifier)) { cachedData[minerIDWithIdentifier] = {totalHashes: 0, goodShares: 0}; } - cachedData[minerIDWithIdentifier].totalHashes += share.shares; - cachedData[minerID].totalHashes += share.shares; - cachedData[minerIDWithIdentifier].goodShares += 1; - cachedData[minerID].goodShares += 1; + cachedData[minerIDWithIdentifier].totalHashes += share.raw_shares; + cachedData[minerID].totalHashes += share.raw_shares; + const share_num = typeof(share.share_num) !== 'undefined' && share.share_num ? share.share_num : 1; + cachedData[minerIDWithIdentifier].goodShares += share_num; + cachedData[minerID].goodShares += share_num; } else { console.error("Error in share parser: " + JSON.stringify(share)); } @@ -248,17 +258,23 @@ function Database(){ txn.putString(global.database.cacheDB, key, JSON.stringify(cachedData[key])); } else { let json_cache = JSON.parse(cacheStore); - if (json_cache.hasOwnProperty('totalHashes')){ + if (json_cache.hasOwnProperty('totalHashes')) { // cachedData.totalHashes is always there for global and miners stats json_cache.totalHashes += cachedData[key].totalHashes; - json_cache.goodShares += cachedData[key].goodShares; } else { json_cache.totalHashes = cachedData[key].totalHashes; - json_cache.goodShares = cachedData[key].goodShares; } - if (cachedData[key].hasOwnProperty('roundHashes') && json_cache.hasOwnProperty('roundHashes')){ - json_cache.roundHashes += cachedData[key].roundHashes; - } else { - json_cache.roundHashes = cachedData[key].roundHashes; + if (cachedData[key].hasOwnProperty('goodShares')) { + if (json_cache.hasOwnProperty('goodShares')) { + json_cache.goodShares += cachedData[key].goodShares; + } else { + json_cache.goodShares = cachedData[key].goodShares; + } + } else if (cachedData[key].hasOwnProperty('roundHashes')) { + if (json_cache.hasOwnProperty('roundHashes')) { + json_cache.roundHashes += cachedData[key].roundHashes; + } else { + json_cache.roundHashes = cachedData[key].roundHashes; + } } txn.putString(global.database.cacheDB, key, JSON.stringify(json_cache)); } @@ -267,10 +283,10 @@ function Database(){ let blocksSeen = 0; for (let key in shares){ if (shares.hasOwnProperty(key)){ - blocksSeen += 1; + ++ blocksSeen; let sharesSeen = 0; shares[key].forEach(function(final_share){ // jshint ignore:line - sharesSeen += 1; + ++ sharesSeen; try { txn.putBinary(global.database.shareDB, parseInt(key), global.protos.Share.encode(final_share)); } catch (e) { @@ -297,39 +313,16 @@ function Database(){ minerID = minerID + '.' + share.paymentID; } let minerIDWithIdentifier = minerID + "_" + share.identifier; - this.incrementCacheData(minerIDWithIdentifier, [{location: 'badShares', value: 1}]); - this.incrementCacheData(minerID, [{location: 'badShares', value: 1}]); + this.incrementCacheData(minerIDWithIdentifier, [{location: 'badShares', value: share.count ? share.count : 1}]); + this.incrementCacheData(minerID, [{location: 'badShares', value: share.count ? share.count : 1}]); callback(true); } catch (e){ - console.error("Ran into an error string an invalid share. Damn!"); + console.error("Ran into an error storing an invalid share. Damn!"); callback(false); } }; - this.getLastBlock = function(blockType){ - this.refreshEnv(); - debug("Getting the last block for: "+ blockType); - let txn = this.env.beginTxn({readOnly: true}); - let cursor = new this.lmdb.Cursor(txn, this.blockDB); - let highestBlock = 0; - for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { - cursor.getCurrentBinary(function(key, data){ // jshint ignore:line - let blockData = global.protos.Block.decode(data); - if (blockData.poolType === blockType || typeof(blockType) === 'undefined'){ - if (found > highestBlock){ - highestBlock = found; - } - } - }); - } - cursor.close(); - txn.commit(); - debug("Done getting the last block for: "+ blockType + " height of: "+ highestBlock); - return highestBlock; - }; - this.getBlockByID = function(blockID){ - this.refreshEnv(); debug("Getting the data for blockID: " + blockID); let txn = this.env.beginTxn({readOnly: true}); let data = txn.getBinary(this.blockDB, blockID); @@ -343,59 +336,58 @@ function Database(){ return blockData; }; - this.calculateShares = function(blockData, blockHeight){ - debug("Calculating shares for "+ blockData.hash); - this.refreshEnv(); - let lastBlock = this.getLastBlock(blockData.poolType); - let shareCount = 0; - range.range(lastBlock+1, blockHeight+1).forEach(function (blockID) { - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - for (let found = (cursor.goToRange(blockID) === blockID); found; found = cursor.goToNextDup()) { - cursor.getCurrentBinary(function(key, data) { // jshint ignore:line - try{ - let shareData = global.protos.Share.decode(data); - if (shareData.poolType === blockData.poolType){ - shareCount = shareCount + shareData.shares; - } - } catch(e){ - console.error("Invalid share"); - } - }); - } - cursor.close(); - txn.commit(); - }); - blockData.shares = shareCount; - debug("Share calculator for "+ blockData.hash + " complete, found " + shareCount + " shares."); - return global.protos.Block.encode(blockData); - }; + // hash -> time + let orphanBlocks = {}; this.storeBlock = function(blockId, blockData, callback){ - this.refreshEnv(); try{ let blockDataDecoded = global.protos.Block.decode(blockData); global.coinFuncs.getBlockHeaderByHash(blockDataDecoded.hash, function(err, header){ + // after 5 minutes of submit attempts finally cosider this block as orphan + if (err && header) { + const is_orphan1 = header.orphan_status && header.orphan_status === true; + const is_orphan2 = header.error && typeof(header.error.message) === 'string' && ( + header.error.message.indexOf("can't get block by hash") > -1 || + header.error.message.indexOf("hash wasn't found") > -1 || + header.error.message.indexOf("Transaction not found") > -1 + ); + if (is_orphan1 || is_orphan2) { + let time_now = Date.now(); + if (blockDataDecoded.hash in orphanBlocks) { + if (time_now - orphanBlocks[blockDataDecoded.hash] > 10*60*1000) { + console.log("Stopped attempts to get block reward for " + blockDataDecoded.hash); + err = false; + header = {}; + header.reward = 0; + blockDataDecoded.valid = false; + blockDataDecoded.unlocked = true; + } + } else { + console.log("Started attempts to store possibly orphan block " + blockDataDecoded.hash); + orphanBlocks[blockDataDecoded.hash] = time_now; + } + } + } if (err || typeof(header) === 'undefined' || !header){ - return callback(false); + setTimeout(function () { return callback(false) }, 30*1000); + return; } blockDataDecoded.value = header.reward; - blockData = global.database.calculateShares(blockDataDecoded, blockId); + //blockData = global.database.calculateShares(blockDataDecoded, blockId); + let shares = global.database.getCache(poolTypeStr(blockDataDecoded.poolType) + "_stats2"); + blockDataDecoded.shares = shares ? shares.roundHashes : 0; + blockData = global.protos.Block.encode(blockDataDecoded); let txn = global.database.env.beginTxn(); + // this is very rare case if block with smae height is found by several leafs (one that is stored later should be orphan with 0 reward) + if (txn.getBinary(global.database.blockDB, blockId) !== null) { + txn.abort(); + console.error("Can't store already stored block with " + blockId.toString() + " key and " + header.reward.toString() + " value: " + JSON.stringify(blockDataDecoded)); + return callback(true); + } txn.putBinary(global.database.blockDB, blockId, blockData); txn.commit(); - global.database.incrementCacheData('global_stats', [{location: 'roundHashes', value: false}]); - switch (blockDataDecoded.poolType) { - case global.protos.POOLTYPE.PPLNS: - global.database.incrementCacheData('pplns_stats', [{location: 'roundHashes', value: false}]); - break; - case global.protos.POOLTYPE.PPS: - global.database.incrementCacheData('pps_stats', [{location: 'roundHashes', value: false}]); - break; - case global.protos.POOLTYPE.SOLO: - global.database.incrementCacheData('solo_stats', [{location: 'roundHashes', value: false}]); - break; - } + global.database.incrementCacheData('global_stats2', [{location: 'roundHashes', value: false}]); + global.database.incrementCacheData(poolTypeStr(blockDataDecoded.poolType) + '_stats2', [{location: 'roundHashes', value: false}]); return callback(true); }); } catch (e) { @@ -404,33 +396,159 @@ function Database(){ } }; - this.fixBlockShares = function(blockId){ - let txn = global.database.env.beginTxn(); - let blockData = txn.getBinary(global.database.blockDB, blockId); - txn.abort(); - let blockDataDecoded = global.protos.Block.decode(blockData); - global.coinFuncs.getBlockHeaderByHash(blockDataDecoded.hash, function(err, header){ - if (err === null) { + let potentiallyBadBlocks = {}; // port -> block hash that has issues that will move into badBlocks after we will find good block for same port + let badBlocks = {}; // block hashes that we just start ignore (can't find incoming wallet tx) + let busyPorts = {}; // ports that are alredy have active getPortBlockHeaderByHash request + + this.storeAltBlock = function(blockId, blockData, callback){ + try{ + let blockDataDecoded = global.protos.AltBlock.decode(blockData); + if (blockDataDecoded.port in busyPorts) { + console.error("Pausing altblock with " + blockDataDecoded.port.toString() + " port and " + blockDataDecoded.height.toString() + " height processing"); + setTimeout(function () { return callback(false) }, 30*1000); + return; + } + busyPorts[blockDataDecoded.port] = 1; + global.coinFuncs.getPortBlockHeaderByHash(blockDataDecoded.port, blockDataDecoded.hash, function(err, header){ + delete busyPorts[blockDataDecoded.port]; + // after 5 minutes of submit attempts finally cosider this block as orphan + let is_orphan = false; + if (err && header) { + const is_orphan1 = header.orphan_status && header.orphan_status === true; + const is_orphan2 = header.error && typeof(header.error.message) === 'string' && ( + header.error.message.indexOf("can't get block by hash") > -1 || + header.error.message.indexOf("Requested hash wasn't found in main blockchain") > -1 + ); + const is_orphan3 = header.topoheight && header.topoheight === -1; + if (is_orphan1 || is_orphan2 || is_orphan3) { + let time_now = Date.now(); + if (blockDataDecoded.hash in orphanBlocks) { + if (time_now - orphanBlocks[blockDataDecoded.hash] > 10*60*1000) { + is_orphan = true; + console.log("Stopped attempts to get block reward for " + blockDataDecoded.hash); + err = false; + header = {}; + header.reward = 0; + blockDataDecoded.valid = false; + blockDataDecoded.unlocked = true; + } + } else { + console.log("Started attempts to store possibly orphan block " + blockDataDecoded.hash); + orphanBlocks[blockDataDecoded.hash] = time_now; + setTimeout(function () { return callback(false) }, 30*1000); + return; + } + } + } + if (blockDataDecoded.port == 20206 && header.depth < 30) { + console.log("Delaying " + blockDataDecoded.port + " port block hash " + blockDataDecoded.hash); + setTimeout(function () { return callback(false) }, 30*1000); + return; + } + if (err || typeof(header) === 'undefined' || !header || !header.reward) { // bad block and not orphan + if (blockDataDecoded.hash in badBlocks) { + console.error("Invalidating " + blockDataDecoded.port + " port block hash " + blockDataDecoded.hash); + return callback(true); + } + if (!(blockDataDecoded.port in potentiallyBadBlocks)) potentiallyBadBlocks[blockDataDecoded.port] = {}; + potentiallyBadBlocks[blockDataDecoded.port][blockDataDecoded.hash] = 1; + setTimeout(function () { return callback(false) }, 30*1000); + return; + } + if (!is_orphan) { // now we found good block (not orphan) and we can move potentiallyBadBlocks to badBlocks + if (blockDataDecoded.port in potentiallyBadBlocks) { + for (let hash in potentiallyBadBlocks[blockDataDecoded.port]) { + console.log("Allowing bad " + blockDataDecoded.port + " port block hash " + hash); + badBlocks[hash] = 1; + } + delete potentiallyBadBlocks[blockDataDecoded.port]; + } + } blockDataDecoded.value = header.reward; - blockData = global.database.calculateShares(blockDataDecoded, blockId); + blockDataDecoded.pay_value = 0; + //blockData = global.database.calculateShares(blockDataDecoded, blockId); + let port_suffix = "_" + blockDataDecoded.port.toString(); + let shares = global.database.getCache(poolTypeStr(blockDataDecoded.poolType) + "_stats2" + port_suffix); + blockDataDecoded.shares = shares ? shares.roundHashes : 0; + blockData = global.protos.AltBlock.encode(blockDataDecoded); + if (global.database.isAltBlockInDB(blockDataDecoded.port, blockDataDecoded.height)) { + console.error("Can't store already stored altblock with " + blockDataDecoded.port.toString() + " port and " + blockDataDecoded.height.toString() + " height and " + header.reward.toString() + " value: " + JSON.stringify(blockDataDecoded)); + return callback(true); + } let txn = global.database.env.beginTxn(); - txn.putBinary(global.database.blockDB, blockId, blockData); + let blockData2; + while ((blockData2 = txn.getBinary(global.database.altblockDB, blockId)) !== null) { + const blockDataDecoded2 = global.protos.AltBlock.decode(blockData2); + if (blockDataDecoded2.hash === blockDataDecoded.hash) { + txn.abort(); + console.error("Can't store already stored altblock with " + blockDataDecoded.hash.toString() + " hash: " + JSON.stringify(blockDataDecoded)); + return callback(true); + } + console.error("Can't store altblock with " + blockId.toString() + " key, trying to increment it"); + ++ blockId; + } + txn.putBinary(global.database.altblockDB, blockId, blockData); txn.commit(); - } - }); + global.database.incrementCacheData('global_stats2' + port_suffix, [{location: 'roundHashes', value: false}]); + global.database.incrementCacheData(poolTypeStr(blockDataDecoded.poolType) + '_stats2' + port_suffix, [{location: 'roundHashes', value: false}]); + return callback(true); + }); + } catch (e) { + console.error("ERROR IN STORING BLOCK. LOOK INTO ME PLZ: " + JSON.stringify(e)); + throw new Error("Error in block storage"); + } }; this.invalidateBlock = function(blockId){ - this.refreshEnv(); let txn = this.env.beginTxn(); let blockData = global.protos.Block.decode(txn.getBinary(this.blockDB, blockId)); blockData.valid = false; + blockData.unlocked = true; txn.putBinary(this.blockDB, blockId, global.protos.Block.encode(blockData)); txn.commit(); }; + this.invalidateAltBlock = function(blockId){ + let txn = this.env.beginTxn(); + let blockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, blockId)); + blockData.valid = false; + blockData.unlocked = true; + txn.putBinary(this.altblockDB, blockId, global.protos.AltBlock.encode(blockData)); + txn.commit(); + }; + + this.changeAltBlockPayStageStatus = function(blockId, pay_stage, pay_status){ + let txn = this.env.beginTxn(); + let blockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, blockId)); + blockData.pay_stage = pay_stage; + blockData.pay_status = pay_status; + txn.putBinary(this.altblockDB, blockId, global.protos.AltBlock.encode(blockData)); + txn.commit(); + }; + + this.moveAltBlockReward = function(srcBlockId, dstBlockId, srcAmount){ + let txn = this.env.beginTxn(); + let srcBlockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, srcBlockId)); + let dstBlockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, dstBlockId)); + dstBlockData.value += srcAmount; + srcBlockData.value = 0; + srcBlockData.pay_stage = "Paid by other block"; + srcBlockData.pay_status = "Will be paid by block " + dstBlockData.hash + " on " + dstBlockData.height + " height"; + srcBlockData.unlocked = true; + txn.putBinary(this.altblockDB, srcBlockId, global.protos.AltBlock.encode(srcBlockData)); + txn.putBinary(this.altblockDB, dstBlockId, global.protos.AltBlock.encode(dstBlockData)); + txn.commit(); + }; + + this.changeAltBlockPayValue = function(blockId, pay_value){ + let txn = this.env.beginTxn(); + let blockData = global.protos.AltBlock.decode(txn.getBinary(this.altblockDB, blockId)); + blockData.pay_value = pay_value; + txn.putBinary(this.altblockDB, blockId, global.protos.AltBlock.encode(blockData)); + txn.commit(); + }; + this.getValidLockedBlocks = function(){ - this.refreshEnv(); let txn = this.env.beginTxn({readOnly: true}); let cursor = new this.lmdb.Cursor(txn, this.blockDB); let blockList = []; @@ -448,8 +566,42 @@ function Database(){ return blockList; }; + this.getValidLockedAltBlocks = function(){ + let txn = this.env.beginTxn({readOnly: true}); + let cursor = new this.lmdb.Cursor(txn, this.altblockDB); + let blockList = []; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.valid === true && blockData.unlocked === false){ + blockData.id = key; + blockList.push(blockData); + } + }); + } + cursor.close(); + txn.commit(); + return blockList; + }; + + this.isAltBlockInDB = function(port, height){ + let txn = this.env.beginTxn({readOnly: true}); + let cursor = new this.lmdb.Cursor(txn, this.altblockDB); + let isBlockFound = false; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.port === port && blockData.height === height){ + isBlockFound = true; + } + }); + } + cursor.close(); + txn.commit(); + return isBlockFound; + }; + this.unlockBlock = function(blockHex){ - this.refreshEnv(); let txn = this.env.beginTxn(); let cursor = new this.lmdb.Cursor(txn, this.blockDB); for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { @@ -467,21 +619,63 @@ function Database(){ txn.commit(); }; - this.lockBlock = function(blockId){ + this.unlockAltBlock = function(blockHex){ + let txn = this.env.beginTxn(); + let cursor = new this.lmdb.Cursor(txn, this.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + let altblockDB = this.altblockDB; + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash === blockHex){ + blockData.unlocked = true; + txn.putBinary(altblockDB, key, global.protos.AltBlock.encode(blockData)); + } + }); + altblockDB = null; + } + cursor.close(); + txn.commit(); + }; + + this.payReadyBlock = function(blockHex){ + let txn = this.env.beginTxn(); + let cursor = new this.lmdb.Cursor(txn, this.blockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + let blockDB = this.blockDB; + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.Block.decode(data); + if (blockData.hash === blockHex){ + blockData.pay_ready = true; + txn.putBinary(blockDB, key, global.protos.Block.encode(blockData)); + } + }); + blockDB = null; + } + cursor.close(); + txn.commit(); + }; + + this.payReadyAltBlock = function(blockHex){ let txn = this.env.beginTxn(); - let blockProto = txn.getBinary(this.blockDB, blockId); - if (blockProto !== null){ - let blockData = global.protos.Block.decode(blockProto); - blockData.unlocked = false; - txn.putBinary(this.blockDB, blockId, global.protos.Block.encode(blockData)); + let cursor = new this.lmdb.Cursor(txn, this.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + let altblockDB = this.altblockDB; + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash === blockHex){ + blockData.pay_ready = true; + txn.putBinary(altblockDB, key, global.protos.AltBlock.encode(blockData)); + } + }); + altblockDB = null; } + cursor.close(); txn.commit(); }; this.getCache = function(cacheKey){ debug("Getting Key: "+cacheKey); try { - this.refreshEnv(); let txn = this.env.beginTxn({readOnly: true}); let cached = txn.getString(this.cacheDB, cacheKey); txn.abort(); @@ -497,7 +691,6 @@ function Database(){ this.setCache = function(cacheKey, cacheData){ debug("Setting Key: "+cacheKey+ " Data: " + JSON.stringify(cacheData)); - this.refreshEnv(); let txn = this.env.beginTxn(); txn.putString(this.cacheDB, cacheKey, JSON.stringify(cacheData)); txn.commit(); @@ -507,72 +700,177 @@ function Database(){ let txn = this.env.beginTxn(); txn.putString(this.cacheDB, 'cacheUpdate', 'cacheUpdate'); txn.commit(); + //let size = 0; txn = this.env.beginTxn(); - for (let key in cacheUpdates){ - if (cacheUpdates.hasOwnProperty(key)){ - txn.putString(this.cacheDB, key, JSON.stringify(cacheUpdates[key])); - } + for (const [key, value] of Object.entries(cacheUpdates)) { + const value_str = JSON.stringify(value); + txn.putString(this.cacheDB, key, value_str); + //size += key.length + value_str.length; } txn.del(this.cacheDB, 'cacheUpdate'); txn.commit(); + //this.env.sync(function() { + //console.log("Wrote " + size + " bytes to LMDB"); + //}); }; - this.cleanShareDB = function() { + this.getOldestLockedBlockHeight = function(){ /* - This function takes the difficulty of the current block, and the last PPS block. If it's 0, save everything, - UNLESS global.config.pps.enable is FALSE, then feel free to trash it. - Due to LMDB under current config, we must delete entire keys, due to this, we save diff * shareMultiLog * 1.5 - global.config.pplns.shareMultiLog should be at least 1.5x your shareMulti, in case of diff spikiing - */ - let lastPPSBlock = this.getLastBlock(); - if (global.config.pps.enable){ - lastPPSBlock = this.getLastBlock(global.protos.POOLTYPE.PPS); - if (lastPPSBlock === 0){ - return; + 6-29-2017 - Snipa - + This function returns a decompressed block proto for the first locked block in the system as part of the + share depth functions. DO NOT BLINDLY REPLACE getLastBlock WITH THIS FUNCTION. + */ + debug("Getting the oldest locked block in the system"); + + let oldestLockedBlockHeight = null; + + let txn = this.env.beginTxn({readOnly: true}); + + { let cursor = new this.lmdb.Cursor(txn, this.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.unlocked === false && blockData.pay_ready !== true){ + if (oldestLockedBlockHeight === null || oldestLockedBlockHeight > blockData.anchor_height) { + oldestLockedBlockHeight = blockData.anchor_height; + } + } + }); } + cursor.close(); + } + + { let cursor = new this.lmdb.Cursor(txn, this.blockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + if (oldestLockedBlockHeight !== null && oldestLockedBlockHeight <= key) return; + let blockData = global.protos.Block.decode(data); + if (blockData.unlocked === false && blockData.pay_ready !== true) { + oldestLockedBlockHeight = key; + } + }); + } + cursor.close(); + } + + txn.commit(); + + if (oldestLockedBlockHeight !== null) { + console.log("Got the oldest locked block in the system at height: " + oldestLockedBlockHeight.toString()); + } else { + console.log("There are no locked blocks in the system. Woohoo!"); } - let lastPPLNSBlock = this.getLastBlock(global.protos.POOLTYPE.PPLNS); - debug("Last PPS block: "+lastPPSBlock); - // Hopping into async, we need the current block height to know where to start our indexing... + return oldestLockedBlockHeight; + }; + + this.cleanShareDB = function() { + /* + This function takes the difficulty of the current block, and the last PPS block. If it's 0, save everything, + UNLESS global.config.pps.enable is FALSE, then feel free to trash it. + Due to LMDB under current config, we must delete entire keys, due to this, we save diff * shareMulti * 2 + 6/29/2017 - Fixed bug with the way the system got blocks. getLastBlock gets the most recent block. + getOldestLockedBlock gets the oldest block in the system that's locked. This ensures that shares for that block + can't be destroyed, and that there's enough depth past that point to ensure the system will have the ability + to make payouts based on the shareMultiLog. Thanks suhz for finding this. Sorry it hit your aeon pool hard. + :( -- Snipa + If current_height - global.payout.blocksRequired > lastLockedBlock, then set the scan start to + current_height - global.payout.blocksRequired - 1 so that we have the full block in case of PPS. + Otherwise, use the lastPPLNSBlock as the scan start. There we go. Stupid logic! + Math check! + cur_height = 100, blocksRequired=20, lastPPLNSLockedBlock.height=90 + In this case, the functional depth required for SOLO is 80 - 1, giving us 79 as our start + cur_height = 100, blocksRequired=20, lastPPLNSLockedBlock.height=70 + In this case, the PPLNS locked block is older than the current height - the required amount, so start is 70. + PPS height no longer matters! Yay! + Solo really doesn't matter, as block finder gets everything. + If there is no valid locked block to start from, aka all blocks are unlocked, then scan from the current height + of the chain, as there's no way for the system to have older blocks. We only need to save extra in the case + where there's unlocked blocks. A find on the current block will have enough depth as long as the saves are + correct. This will cause the system to clean up shares massively when there are no unlocked blocks. + */ + if (cleanShareInProgress) { + console.error("CleanShareDB already running"); + ++cleanShareStuckCount; + if (cleanShareStuckCount > 5) global.support.sendEmail(global.config.general.adminEmail,"LongRunner stuck",cleanShareStuckCount); + return; // already running + } + cleanShareInProgress = true; + let oldestLockedBlockHeight = this.getOldestLockedBlockHeight(); async.waterfall([ function(callback){ + if (oldestLockedBlockHeight === null) { + callback(null, null, null); + } else { + global.coinFuncs.getBlockHeaderByID(oldestLockedBlockHeight, (err, result) => { + if (err !== null) { + console.error("Can't get block with " + oldestLockedBlockHeight + " height"); + return callback(true); + } + callback(null, oldestLockedBlockHeight, result.difficulty); + }); + } + }, + function(oldestLockedBlockHeight, oldestLockedBlockDifficulty, callback){ global.coinFuncs.getLastBlockHeader(function(err, body){ - if (err) { - return callback(true, body); + if (err !== null) { + console.error("Last block header request failed!"); + return callback(true); + } + if (oldestLockedBlockHeight === null){ + /* + If there's no locked blocks, then allow the system to scan from the PPS depth downwards if PPS + is enabled. + Save enough shares so that the diff * share multi * 30% for buffer. + */ + if (global.config.pps.enable){ + // If PPS is enabled, we scan for new blocks at cur height - blocksRequired/2. + // We need to save shares back that far at the least. + callback(null, body.height - Math.floor(global.config.payout.blocksRequired/2), Math.floor(body.difficulty * global.config.pplns.shareMulti * 2)); + } else { + // Otherwise, we can just start from the current height. Woo! + callback(null, body.height, Math.floor(body.difficulty * global.config.pplns.shareMulti * 2)); + } + + } else { + console.log("Block depth to keep is " + (body.height - oldestLockedBlockHeight)); + if (body.height - oldestLockedBlockHeight > global.config.general.blockCleanWarning) { + global.support.sendEmail(global.config.general.adminEmail, "longRunner module can not clean DB good enough", "longRunner can not clean " + (body.height - oldestLockedBlockHeight) + " block from DB!"); + } + /* + Otherwise, start the scan from the oldest locked block downwards. + This protects against the blockManager being messed up and not unlocking blocks. + This will ensure that enough shares are in place to unlock all blocks. + If the block is Solo, PPLNS or PPS, it doesn't matter. + */ + if (global.config.pps.enable && oldestLockedBlockHeight > body.height - Math.floor(global.config.payout.blocksRequired/2)) { + // If PPS is enabled, and the oldestLockedBlockHeight > the PPS minimum, start from the PPS minimum. + callback(null, body.height - Math.floor(global.config.payout.blocksRequired/2), Math.floor(body.difficulty * global.config.pplns.shareMulti * 2)); + } else { + // If PPS isn't enabled, or the oldestLockedBlockHeight < the PPS minimum, then start from there. + callback(null, oldestLockedBlockHeight, Math.floor(oldestLockedBlockDifficulty * global.config.pplns.shareMulti * 2)); + } } - return callback(null, body.height, Math.floor(body.difficulty * 1.5 * global.config.pplns.shareMultiLog)); }); }, function (lastBlock, difficulty, callback) { let shareCount = 0; - let ppsFound = false; let pplnsFound = false; - let blockList = []; - debug("Scanning from: "+lastBlock + " for more than: " + difficulty + " shares"); - range.range(0, lastBlock+1).forEach(function (blockID) { - blockID = (blockID - lastBlock+1) * -1; - if (blockID < 0){ - return; - } + let blockSet = {}; + console.log("Scanning from: "+lastBlock + " for more than: " + difficulty + " shares"); + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + range.range(lastBlock-1, 0, -1).forEach(function (blockID) { debug("Scanning block: " + blockID); - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - for (let found = (cursor.goToRange(blockID) === blockID); found; found = cursor.goToNextDup()) { - if (ppsFound && pplnsFound){ - cursor.getCurrentBinary(function(key, data) { // jshint ignore:line - if (blockList.indexOf(key) === -1){ - blockList.push(key); - } - }); + for (let found = (cursor.goToRange(parseInt(blockID)) === blockID); found; found = cursor.goToNextDup()) { + if (pplnsFound) { + blockSet[blockID] = 1; + break; } else { cursor.getCurrentBinary(function(key, data) { // jshint ignore:line - if (key < lastPPSBlock){ - ppsFound = true; - } try{ let shareData = global.protos.Share.decode(data); if (shareData.poolType === global.protos.POOLTYPE.PPLNS){ - shareCount = shareCount + shareData.shares; + shareCount += shareData.shares2; } } catch(e){ console.error("Invalid share"); @@ -580,50 +878,59 @@ function Database(){ }); if (shareCount >= difficulty){ pplnsFound = true; + console.log("Found the first block to be deleted at " + blockID + " height"); + break; } } } - cursor.close(); - txn.abort(); }); - callback(null, blockList); + cursor.close(); + txn.abort(); + console.log("Scan finished"); + callback(null, Array.from(Object.keys(blockSet))); } ], function(err, data){ - if (err === null && global.config.general.blockCleaner === true){ + if (err !== null) { + console.error("ERROR with cleaning up because of daemon stuck"); + cleanShareInProgress = false; + return; + } + if (global.config.general.blockCleaner === true){ if(data.length > 0){ - global.database.refreshEnv(); - let blockList = global.database.getBlockList(); - debug("Got the block list"); let totalDeleted = 0; + let totalDeleted2 = 0; + console.log("Block cleaning started: removing " + data.length + " block share records"); + let txn = global.database.env.beginTxn(); data.forEach(function(block){ - if ((blockList.indexOf(block) !== -1 && !blockList.unlocked) || block > lastPPLNSBlock){ - // Don't delete locked blocks. ffs. - // Don't delete blocks that could contain shares. Even if it's unlikely as all getout. - debug("Skipped deleting block: " + block); - return; - } - totalDeleted += 1; - let txn = global.database.env.beginTxn(); - txn.del(global.database.shareDB, block); - txn.commit(); - debug("Deleted block: " + block); + ++ totalDeleted; + ++ totalDeleted2; + debug("Deleted block: " + parseInt(block)); + txn.del(global.database.shareDB, parseInt(block)); + if (totalDeleted2 > 100) { + txn.commit(); + txn = global.database.env.beginTxn(); + totalDeleted2 = 0; + } }); - console.log("Block cleaning enabled. Removed: " +totalDeleted+ " block share records"); + txn.commit(); + console.log("Block cleaning finished: removed " + totalDeleted + " block share records"); } global.database.env.sync(function(){ }); } else { - console.log("Block cleaning disabled. Would of removed: " + JSON.stringify(data)); + console.log("Block cleaning disabled. Would have removed: " + JSON.stringify(data)); } + cleanShareInProgress = false; + cleanShareStuckCount = 0; + console.log("Done cleaning up the share DB"); }); }; +} +process.on('SIGINT', function() { + console.log("Closing DB"); + global.database.env.close(); +}); - this.refreshEnv = function(){}; - - setInterval(function(){ - global.database.dirtyenv = true; - }, 900000); // Set DB env reload for every 15 minutes. -} -module.exports = Database; \ No newline at end of file +module.exports = Database; diff --git a/lib/longRunner.js b/lib/longRunner.js index 4013efba..97090eb6 100644 --- a/lib/longRunner.js +++ b/lib/longRunner.js @@ -1,10 +1,205 @@ "use strict"; +const async = require("async"); + +function cleanCacheDB() { + console.log("Cleaning up the cache DB. Searching for items to delete/update"); + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.cacheDB); + let updated = {}; + let deleted = []; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentString(function(key, data){ // jshint ignore:line + if (key.length < global.config.pool.address.length) return; // min XMR address length + + if (key.includes("identifiers:")) { // remove frozen worker names after 24h + let parts = key.split(/:(.+)/); + let key2 = parts[1]; + + try { + let data2 = JSON.parse(data); + if (data2.length == 0) return; + let isAlive = false; + for (let i in data2) { + let stats = global.database.getCache("stats:" + key2 + "_" + data2[i]); + if (stats && Date.now() - stats.lastHash <= 24*60*60*1000) isAlive = true; + } + if (!isAlive) { + updated[key] = JSON.stringify([]); + } + + } catch (e) { + console.error("Bad cache data with " + key + " key"); + } + + } else if (key.includes("_") && !key.includes("history:") && !key.includes("stats:")) { // remove week old workers + let stats = global.database.getCache("stats:" + key); + if (!stats) return; + if (!global.database.getCache("history:" + key)) return; + if (Date.now() - stats.lastHash > 7*24*60*60*1000) { + deleted.push(key); + deleted.push("history:" + key); + deleted.push("stats:" + key); + } + + } else if (!key.includes("_") && key.includes("stats:")) { // zero frozen account hashrate after 24h + try { + let data2 = JSON.parse(data); + if ((data2.hash || data2.hash2) && Date.now() - data2.lastHash > 24*60*60*1000) { + data2.hash = data2.hash2 = 0; + updated[key] = JSON.stringify(data2); + } + } catch (e) { + console.error("Bad cache data with " + key + " key"); + } + + } + }); + } + + cursor.close(); + txn.commit(); + + console.log("Deleting cache items: " + deleted.length); + + let chunkSize = 0; + txn = global.database.env.beginTxn(); + deleted.forEach(function(key) { + ++ chunkSize; + txn.del(global.database.cacheDB, key); + if (chunkSize > 500) { + txn.commit(); + txn = global.database.env.beginTxn(); + chunkSize = 0; + } + }); + txn.commit(); + + console.log("Updating cache items: " + Object.keys(updated).length); + + chunkSize = 0; + txn = global.database.env.beginTxn(); + for (const [key, value] of Object.entries(updated)) { + ++ chunkSize; + txn.putString(global.database.cacheDB, key, value); + if (chunkSize > 500) { + txn.commit(); + txn = global.database.env.beginTxn(); + chunkSize = 0; + } + } + txn.commit(); +} + +function cleanAltBlockDB() { + console.log("Cleaning up the alt block DB. Searching for items to delete"); + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let deleted = []; + + let block_count = {}; + for (let found = cursor.goToLast(); found; found = cursor.goToPrev()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (!(blockData.port in block_count)) block_count[blockData.port] = 0; + ++ block_count[blockData.port]; + if (blockData.unlocked && (block_count[blockData.port] > 20000 || Date.now() - blockData.timestamp > 3*365*24*60*60*1000)) { + deleted.push(key); + } + }); + } + + cursor.close(); + txn.commit(); + + console.log("Deleting altblock items: " + deleted.length); + + let chunkSize = 0; + txn = global.database.env.beginTxn(); + deleted.forEach(function(key) { + ++ chunkSize; + txn.del(global.database.altblockDB, key); + if (chunkSize > 500) { + txn.commit(); + txn = global.database.env.beginTxn(); + chunkSize = 0; + } + }); + txn.commit(); +} + +//let saw_block_hash_before = {}; + +let cleanBlockBalanceTableQueue = async.queue(function (task, callback) { + global.mysql.query("DELETE FROM block_balance WHERE hex = ?", [task.hex]).then(function () { + return callback(true); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(false); + }); +}, 10); + +setInterval(function(queue_obj){ + if (queue_obj.length()){ + console.log("Remove block balance queue length: " + queue_obj.length()); + } +}, 60*1000, cleanBlockBalanceTableQueue); + +function cleanBlockBalanceTable() { + console.log("Cleaning up the block balance table"); + + let locked_block_hashes = {}; + global.database.getValidLockedBlocks().forEach(function (block) { locked_block_hashes[block.hash] = 1; }); + global.database.getValidLockedAltBlocks().forEach(function (block) { locked_block_hashes[block.hash] = 1; }); + + console.log("Starting cleaning the block balance table. Found " + Object.keys(locked_block_hashes).length + " locked blocks"); + global.mysql.query("SELECT hex FROM paid_blocks WHERE paid_time > (NOW() - INTERVAL 2 DAY)").then(function (rows_keep) { + console.log("Got " + rows_keep.length + " recent blocks"); + rows_keep.forEach(function (row) { locked_block_hashes[row.hex] = 1; }); + let deleted_row_count = 0; + global.mysql.query("SELECT DISTINCT hex FROM block_balance").then(function (rows) { + console.log("Got " + rows.length + " block balance blocks"); + rows.forEach(function (row) { + if (row.hex in locked_block_hashes) return; + //if (row.hex in saw_block_hash_before) { + cleanBlockBalanceTableQueue.push(row, function () {}); + //delete saw_block_hash_before[row.hex]; + ++ deleted_row_count; + //} else { + // saw_block_hash_before[row.hex] = 1; + //} + }); + console.log("Finished preparing the block balance table. Removing " + deleted_row_count + " rows (" + Object.keys(locked_block_hashes).length + " locked)."); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); +} + console.log("Cleaning up the share DB"); global.database.cleanShareDB(); -console.log("Done cleaning up the shareDB"); +cleanCacheDB(); +cleanAltBlockDB(); +cleanBlockBalanceTable(); + +setInterval(function(){ + console.log("Cleaning up the share DB"); + global.database.cleanShareDB(); +}, 4*60*60*1000); + +// clean cache items +setInterval(function(){ + cleanCacheDB(); +}, 24*60*60*1000); + +// clean altblock items +setInterval(function(){ + cleanAltBlockDB(); +}, 7*24*60*60*1000); + +// clean block balance table setInterval(function(){ - console.log("Cleaning up the share DB"); - global.database.cleanShareDB(); - console.log("Done cleaning up the shareDB"); -}, 3600000); \ No newline at end of file + cleanBlockBalanceTable(); +}, 24*60*60*1000); diff --git a/lib/payment_systems/aeon.js b/lib/payment_systems/aeon.js deleted file mode 100644 index 28ba1a91..00000000 --- a/lib/payment_systems/aeon.js +++ /dev/null @@ -1,257 +0,0 @@ -"use strict"; -const async = require("async"); -const debug = require("debug")("payments"); - -let hexChars = new RegExp("[0-9a-f]+"); -let extraPaymentRound = false; -let paymentTimer = null; - -let paymentQueue = async.queue(function (paymentDetails, callback) { - if (paymentTimer !== null){ - clearInterval(paymentTimer); - paymentTimer = null; - } - debug("Making payment based on: " + JSON.stringify(paymentDetails)); - let transferFunc = 'transfer'; - global.support.rpcWallet(transferFunc, paymentDetails, function (body) { - debug("Payment made: " + JSON.stringify(body)); - if (body.hasOwnProperty('error')) { - if (body.error.message === "not enough money"){ - console.error("Issue making payments, not enough money, will try later"); - if(!extraPaymentRound){ - setTimeout(function(){ - makePayments(); - }, global.config.payout.timerRetry * 60 * 1000); - } - extraPaymentRound = true; - return callback(false); - } else { - console.error("Issue making payments" + JSON.stringify(body.error)); - console.error("Will not make more payments until the payment daemon is restarted!"); - //toAddress, subject, body - global.support.sendEmail(global.config.general.adminEmail, "Payment daemon unable to make payment", - "Hello,\r\nThe payment daemon has hit an issue making a payment: " + JSON.stringify(body.error) + - ". Please investigate and restart the payment daemon as appropriate"); - return; - } - } - if (paymentDetails.hasOwnProperty('payment_id')) { - console.log("Payment made to " + paymentDetails.destinations[0].address + " with PaymentID: " + paymentDetails.payment_id + " For: " + global.support.coinToDecimal(paymentDetails.destinations[0].amount) + " XMR with a " + global.support.coinToDecimal(body.result.fee) + " XMR Mining Fee"); - return callback(body.result); - } else { - if (transferFunc === 'transfer') { - console.log("Payment made out to multiple people, total fee: " + global.support.coinToDecimal(body.result.fee) + " XMR"); - } - let intCount = 0; - paymentDetails.destinations.forEach(function (details) { - console.log("Payment made to: " + details.address + " For: " + global.support.coinToDecimal(details.amount) + " XMR"); - intCount += 1; - if (intCount === paymentDetails.destinations.length) { - return callback(body.result); - } - }); - } - }); -}, 1); - -paymentQueue.drain = function(){ - extraPaymentRound = false; - if (global.config.payout.timer > 35791){ - console.error("Payout timer is too high. Please use a value under 35791 to avoid overflows."); - } else { - paymentTimer = setInterval(makePayments, global.config.payout.timer * 60 * 1000); - } - global.database.setCache('lastPaymentCycle', Math.floor(Date.now()/1000)); -}; - -function Payee(amount, address, paymentID, bitcoin) { - this.amount = amount; - this.address = address; - this.paymentID = paymentID; - this.bitcoin = bitcoin; - this.blockID = 0; - this.poolType = ''; - this.transactionID = 0; - this.sqlID = 0; - if (paymentID === null) { - this.id = address; - } else { - this.id = address + "." + paymentID; - } - this.fee = 0; - this.baseFee = global.support.decimalToCoin(global.config.payout.feeSlewAmount); - this.setFeeAmount = function () { - if (this.amount <= global.support.decimalToCoin(global.config.payout.walletMin)) { - this.fee = this.baseFee; - } else if (this.amount <= global.support.decimalToCoin(global.config.payout.feeSlewEnd)) { - let feeValue = this.baseFee / (global.support.decimalToCoin(global.config.payout.feeSlewEnd) - global.support.decimalToCoin(global.config.payout.walletMin)); - this.fee = this.baseFee - ((this.amount - global.support.decimalToCoin(global.config.payout.walletMin)) * feeValue); - } - this.fee = Math.floor(this.fee); - }; - - this.makePaymentWithID = function () { - let paymentDetails = { - destinations: [ - { - amount: this.amount - this.fee, - address: this.address - } - ], - fee: global.config.payout.fee, - unlock_time: global.config.payout.unlock_time, - mixin: global.config.payout.mixIn, - payment_id: this.paymentID - }; - let identifier = this.id; - let amount = this.amount; - let address = this.address; - let paymentID = this.paymentID; - let payee = this; - debug("Payment Details: " + JSON.stringify(paymentDetails)); - paymentQueue.push(paymentDetails, function (body) { - if (typeof body.tx_hash !== 'undefined') { - debug("Successful payment sent to: " + identifier); - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [0, address, paymentID, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, global.config.payout.fee, 1]).then(function (result) { - payee.transactionID = result.insertId; - payee.trackPayment(); - }); - } else { - console.error("Unknown error from the wallet."); - } - }); - }; - - this.makePaymentAsIntegrated = function () { - let paymentDetails = { - destinations: [ - { - amount: this.amount - this.fee, - address: this.address - } - ], - fee: global.config.payout.fee, - unlock_time: global.config.payout.unlock_time, - mixin: global.config.payout.mixIn - }; - let identifier = this.id; - let amount = this.amount; - let address = this.address; - let payee = this; - - debug("Payment Details: " + JSON.stringify(paymentDetails)); - paymentQueue.push(paymentDetails, function (body) { - if (typeof body.tx_hash !== 'undefined') { - debug("Successful payment sent to: " + identifier); - global.mysql.query("INSERT INTO transactions (bitcoin, address, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?)", - [0, address, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, global.config.payout.fee, 1]).then(function (result) { - payee.transactionID = result.insertId; - payee.trackPayment(); - }); - } else { - console.error("Unknown error from the wallet."); - } - }); - }; - - this.trackPayment = function () { - global.mysql.query("UPDATE balance SET amount = amount - ? WHERE id = ?", [this.amount, this.sqlID]); - global.mysql.query("INSERT INTO payments (unlocked_time, paid_time, pool_type, payment_address, transaction_id, bitcoin, amount, payment_id, transfer_fee)" + - " VALUES (now(), now(), ?, ?, ?, ?, ?, ?, ?)", [this.poolType, this.address, this.transactionID, this.bitcoin, this.amount - this.fee, this.paymentID, this.fee]); - }; -} - -function makePayments() { - global.mysql.query("SELECT * FROM balance WHERE amount >= ?", [global.support.decimalToCoin(global.config.payout.walletMin)]).then(function (rows) { - console.log("Loaded all payees into the system for processing"); - let paymentDestinations = []; - let totalAmount = 0; - let roundCount = 0; - let payeeList = []; - let payeeObjects = {}; - rows.forEach(function (row) { - debug("Starting round for: " + JSON.stringify(row)); - let payee = new Payee(row.amount, row.payment_address, row.payment_id, row.bitcoin); - payeeObjects[row.payment_address] = payee; - global.mysql.query("SELECT payout_threshold FROM users WHERE username = ?", [payee.id]).then(function (userRow) { - roundCount += 1; - let threshold = 0; - if (userRow.length !== 0) { - threshold = userRow[0].payout_threshold; - } - payee.poolType = row.pool_type; - payee.sqlID = row.id; - if (payee.poolType === "fees" && payee.address === global.config.payout.feeAddress && payee.amount >= ((global.support.decimalToCoin(global.config.payout.feesForTXN) + global.support.decimalToCoin(global.config.payout.exchangeMin)))) { - debug("This is the fee address internal check for value"); - payee.amount -= global.support.decimalToCoin(global.config.payout.feesForTXN); - } else if (payee.address === global.config.payout.feeAddress && payee.poolType === "fees") { - debug("Unable to pay fee address."); - payee.amount = 0; - } - let remainder = payee.amount % (global.config.payout.denom * global.config.general.sigDivisor); - if (remainder !== 0) { - payee.amount -= remainder; - } - if (payee.amount > threshold) { - payee.setFeeAmount(); - if (payee.bitcoin === 0 && payee.paymentID === null && payee.amount !== 0 && payee.amount > 0 && payee.address.length !== 106) { - debug("Adding " + payee.id + " to the list of people to pay (OG Address). Payee balance: " + global.support.coinToDecimal(payee.amount)); - paymentDestinations.push({amount: payee.amount - payee.fee, address: payee.address}); - totalAmount += payee.amount; - payeeList.push(payee); - } else if ((payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && threshold !== 0)) && payee.bitcoin === 0) { - debug("Adding " + payee.id + " to the list of people to pay (Payment ID Address). Payee balance: " + global.support.coinToDecimal(payee.amount)); - payee.makePaymentWithID(); - } - } - debug("Went: " + roundCount + " With: " + paymentDestinations.length + " Possible destinations and: " + rows.length + " Rows"); - if (roundCount === rows.length && paymentDestinations.length > 0) { - while (paymentDestinations.length > 0) { - let paymentDetails = { - destinations: paymentDestinations.splice(0, global.config.payout.maxPaymentTxns), - mixin: global.config.payout.mixIn, - fee: global.config.payout.fee, - unlock_time: global.config.payout.unlock_time - }; - console.log("Paying out: " + paymentDetails.destinations.length + " people"); - paymentQueue.push(paymentDetails, function (body) { //jshint ignore:line - // This is the only section that could potentially contain multiple txns. Lets do this safely eh? - if (typeof body.tx_hash !== 'undefined') { - debug("Made it to the SQL insert for transactions"); - let totalAmount = 0; - paymentDetails.destinations.forEach(function (payeeItem) { - totalAmount += payeeObjects[payeeItem.address].amount; - totalAmount += payeeObjects[payeeItem.address].fee; - }); - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [0, null, null, totalAmount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, global.config.payout.fee, paymentDetails.destinations.length]).then(function (result) { - paymentDetails.destinations.forEach(function (payeeItem) { - payee = payeeObjects[payeeItem.address]; - payee.transactionID = result.insertId; - payee.trackPayment(); - }); - }); - } else { - console.error("Unknown error from the wallet."); - } - }); - } - } - }); - }); - }); -} - -function init() { - global.support.rpcWallet("store", [], function () { - }); - setInterval(function () { - global.support.rpcWallet("store", [], function () { - }); - }, 60000); - console.log("Setting the payment timer to: " + global.config.payout.timer + " minutes with a: " + global.config.payout.timerRetry + " minute delay if the wallet is out of money"); - makePayments(); -} - -init(); \ No newline at end of file diff --git a/lib/payment_systems/krb.js b/lib/payment_systems/krb.js deleted file mode 100644 index bc907c1a..00000000 --- a/lib/payment_systems/krb.js +++ /dev/null @@ -1,254 +0,0 @@ -"use strict"; -const async = require("async"); -const debug = require("debug")("payments"); - -let hexChars = new RegExp("[0-9a-f]+"); -let extraPaymentRound = false; -let paymentTimer = null; - -let paymentQueue = async.queue(function (paymentDetails, callback) { - if (paymentTimer !== null){ - clearInterval(paymentTimer); - paymentTimer = null; - } - debug("Making payment based on: " + JSON.stringify(paymentDetails)); - let transferFunc = 'transfer'; - global.support.rpcWallet(transferFunc, paymentDetails, function (body) { - debug("Payment made: " + JSON.stringify(body)); - if (body.hasOwnProperty('error')) { - if (body.error.message === "not enough money"){ - console.error("Issue making payments, not enough money, will try later"); - if(!extraPaymentRound){ - setTimeout(function(){ - makePayments(); - }, global.config.payout.timerRetry * 60 * 1000); - } - extraPaymentRound = true; - return callback(false); - } else { - console.error("Issue making payments" + JSON.stringify(body.error)); - console.error("Will not make more payments until the payment daemon is restarted!"); - //toAddress, subject, body - global.support.sendEmail(global.config.general.adminEmail, "Payment daemon unable to make payment", - "Hello,\r\nThe payment daemon has hit an issue making a payment: " + JSON.stringify(body.error) + - ". Please investigate and restart the payment daemon as appropriate"); - return; - } - } - if (paymentDetails.hasOwnProperty('payment_id')) { - console.log("Payment made to " + paymentDetails.destinations[0].address + " with PaymentID: " + paymentDetails.payment_id + " For: " + global.support.coinToDecimal(paymentDetails.destinations[0].amount) + " XMR with a " + global.support.coinToDecimal(body.result.fee) + " XMR Mining Fee"); - return callback(body.result); - } else { - if (transferFunc === 'transfer') { - console.log("Payment made out to multiple people, total fee: " + global.support.coinToDecimal(body.result.fee) + " XMR"); - } - let intCount = 0; - paymentDetails.destinations.forEach(function (details) { - console.log("Payment made to: " + details.address + " For: " + global.support.coinToDecimal(details.amount) + " XMR"); - intCount += 1; - if (intCount === paymentDetails.destinations.length) { - return callback(body.result); - } - }); - } - }); -}, 1); - -paymentQueue.drain = function(){ - extraPaymentRound = false; - if (global.config.payout.timer > 35791){ - console.error("Payout timer is too high. Please use a value under 35791 to avoid overflows."); - } else { - paymentTimer = setInterval(makePayments, global.config.payout.timer * 60 * 1000); - } - global.database.setCache('lastPaymentCycle', Math.floor(Date.now()/1000)); -}; - -function Payee(amount, address, paymentID, bitcoin) { - this.amount = amount; - this.address = address; - this.paymentID = paymentID; - this.bitcoin = bitcoin; - this.blockID = 0; - this.poolType = ''; - this.transactionID = 0; - this.sqlID = 0; - if (paymentID === null) { - this.id = address; - } else { - this.id = address + "." + paymentID; - } - this.fee = 0; - this.baseFee = global.support.decimalToCoin(global.config.payout.feeSlewAmount); - this.setFeeAmount = function () { - if (this.amount <= global.support.decimalToCoin(global.config.payout.walletMin)) { - this.fee = this.baseFee; - } else if (this.amount <= global.support.decimalToCoin(global.config.payout.feeSlewEnd)) { - let feeValue = this.baseFee / (global.support.decimalToCoin(global.config.payout.feeSlewEnd) - global.support.decimalToCoin(global.config.payout.walletMin)); - this.fee = this.baseFee - ((this.amount - global.support.decimalToCoin(global.config.payout.walletMin)) * feeValue); - } - this.fee = Math.floor(this.fee); - }; - - this.makePaymentWithID = function () { - let paymentDetails = { - destinations: [ - { - amount: this.amount - this.fee, - address: this.address - } - ], - priority: global.config.payout.priority, - mixin: global.config.payout.mixIn, - payment_id: this.paymentID - }; - let identifier = this.id; - let amount = this.amount; - let address = this.address; - let paymentID = this.paymentID; - let payee = this; - debug("Payment Details: " + JSON.stringify(paymentDetails)); - paymentQueue.push(paymentDetails, function (body) { - if (body.fee && body.fee > 10) { - debug("Successful payment sent to: " + identifier); - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [0, address, paymentID, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, 1]).then(function (result) { - payee.transactionID = result.insertId; - payee.trackPayment(); - }); - } else { - console.error("Unknown error from the wallet."); - } - }); - }; - - this.makePaymentAsIntegrated = function () { - let paymentDetails = { - destinations: [ - { - amount: this.amount - this.fee, - address: this.address - } - ], - priority: global.config.payout.priority, - mixin: global.config.payout.mixIn - }; - let identifier = this.id; - let amount = this.amount; - let address = this.address; - let payee = this; - - debug("Payment Details: " + JSON.stringify(paymentDetails)); - paymentQueue.push(paymentDetails, function (body) { - if (body.fee && body.fee > 10) { - debug("Successful payment sent to: " + identifier); - global.mysql.query("INSERT INTO transactions (bitcoin, address, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?)", - [0, address, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, 1]).then(function (result) { - payee.transactionID = result.insertId; - payee.trackPayment(); - }); - } else { - console.error("Unknown error from the wallet."); - } - }); - }; - - this.trackPayment = function () { - global.mysql.query("UPDATE balance SET amount = amount - ? WHERE id = ?", [this.amount, this.sqlID]); - global.mysql.query("INSERT INTO payments (unlocked_time, paid_time, pool_type, payment_address, transaction_id, bitcoin, amount, payment_id, transfer_fee)" + - " VALUES (now(), now(), ?, ?, ?, ?, ?, ?, ?)", [this.poolType, this.address, this.transactionID, this.bitcoin, this.amount - this.fee, this.paymentID, this.fee]); - }; -} - -function makePayments() { - global.mysql.query("SELECT * FROM balance WHERE amount >= ?", [global.support.decimalToCoin(global.config.payout.walletMin)]).then(function (rows) { - console.log("Loaded all payees into the system for processing"); - let paymentDestinations = []; - let totalAmount = 0; - let roundCount = 0; - let payeeList = []; - let payeeObjects = {}; - rows.forEach(function (row) { - debug("Starting round for: " + JSON.stringify(row)); - let payee = new Payee(row.amount, row.payment_address, row.payment_id, row.bitcoin); - payeeObjects[row.payment_address] = payee; - global.mysql.query("SELECT payout_threshold FROM users WHERE username = ?", [payee.id]).then(function (userRow) { - roundCount += 1; - let threshold = 0; - if (userRow.length !== 0) { - threshold = userRow[0].payout_threshold; - } - payee.poolType = row.pool_type; - payee.sqlID = row.id; - if (payee.poolType === "fees" && payee.address === global.config.payout.feeAddress && payee.amount >= ((global.support.decimalToCoin(global.config.payout.feesForTXN) + global.support.decimalToCoin(global.config.payout.exchangeMin)))) { - debug("This is the fee address internal check for value"); - payee.amount -= global.support.decimalToCoin(global.config.payout.feesForTXN); - } else if (payee.address === global.config.payout.feeAddress && payee.poolType === "fees") { - debug("Unable to pay fee address."); - payee.amount = 0; - } - let remainder = payee.amount % (global.config.payout.denom * global.config.general.sigDivisor); - if (remainder !== 0) { - payee.amount -= remainder; - } - if (payee.amount > threshold) { - payee.setFeeAmount(); - if (payee.bitcoin === 0 && payee.paymentID === null && payee.amount !== 0 && payee.amount > 0 && payee.address.length !== 106) { - debug("Adding " + payee.id + " to the list of people to pay (OG Address). Payee balance: " + global.support.coinToDecimal(payee.amount)); - paymentDestinations.push({amount: payee.amount - payee.fee, address: payee.address}); - totalAmount += payee.amount; - payeeList.push(payee); - } else if ((payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && threshold !== 0)) && payee.bitcoin === 0) { - debug("Adding " + payee.id + " to the list of people to pay (Payment ID Address). Payee balance: " + global.support.coinToDecimal(payee.amount)); - payee.makePaymentWithID(); - } - } - debug("Went: " + roundCount + " With: " + paymentDestinations.length + " Possible destinations and: " + rows.length + " Rows"); - if (roundCount === rows.length && paymentDestinations.length > 0) { - while (paymentDestinations.length > 0) { - let paymentDetails = { - destinations: paymentDestinations.splice(0, global.config.payout.maxPaymentTxns), - priority: global.config.payout.priority, - mixin: global.config.payout.mixIn - }; - console.log("Paying out: " + paymentDetails.destinations.length + " people"); - paymentQueue.push(paymentDetails, function (body) { //jshint ignore:line - // This is the only section that could potentially contain multiple txns. Lets do this safely eh? - if (body.fee && body.fee > 10) { - debug("Made it to the SQL insert for transactions"); - let totalAmount = 0; - paymentDetails.destinations.forEach(function (payeeItem) { - totalAmount += payeeObjects[payeeItem.address].amount; - totalAmount += payeeObjects[payeeItem.address].fee; - }); - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [0, null, null, totalAmount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, paymentDetails.destinations.length]).then(function (result) { - paymentDetails.destinations.forEach(function (payeeItem) { - payee = payeeObjects[payeeItem.address]; - payee.transactionID = result.insertId; - payee.trackPayment(); - }); - }); - } else { - console.error("Unknown error from the wallet."); - } - }); - } - } - }); - }); - }); -} - -function init() { - global.support.rpcWallet("store", [], function () { - }); - setInterval(function () { - global.support.rpcWallet("store", [], function () { - }); - }, 60000); - console.log("Setting the payment timer to: " + global.config.payout.timer + " minutes with a: " + global.config.payout.timerRetry + " minute delay if the wallet is out of money"); - makePayments(); -} - -init(); \ No newline at end of file diff --git a/lib/payment_systems/xmr.js b/lib/payment_systems/xmr.js index 596397c0..564a1657 100644 --- a/lib/payment_systems/xmr.js +++ b/lib/payment_systems/xmr.js @@ -1,480 +1,88 @@ "use strict"; -const shapeshift = require('shapeshift.io'); const async = require("async"); const debug = require("debug")("payments"); const request = require('request-json'); const range = require('range'); +const sprintf = require("sprintf-js").sprintf; let hexChars = new RegExp("[0-9a-f]+"); -let bestExchange = global.config.payout.bestExchange; -let xmrAPIClient = request.createClient('https://xmr.to/api/v1/xmr2btc/'); -let extraPaymentRound = false; -let paymentTimer = null; -let shapeshiftQueue = async.queue(function (task, callback) { - // Amount needs to be shifted in as a non-completed value, as the wallet will only take non-complete values.. - let amount = task.amount - task.fee; - // Address is the destination address IN BTC. - let address = task.address; - // PaymentIDs are the paymentID's to flag as paid by this transaction. - // Should be a massive list of ID's so we can bulk-update them, by merging them with 's. - // Here we go! General process: Scan shapeshift for valid amounts of funds to xfer around. - // Once there's enough funds, then we active txn - // Do a wallet call to xfer. - // Setup a monitor on the transaction - async.waterfall([ - function (intCallback) { - // Verify if the coin is active in ShapeShift first. - shapeshift.coins(function (err, coinData) { - if (err) { - intCallback(err); - } else if (!coinData.hasOwnProperty(global.config.general.coinCode) || coinData[global.config.general.coinCode].status !== "available") { - intCallback("Coin " + global.config.general.coinCode + " Is not available at this time on shapeshift."); - } else { - intCallback(null); - } - }); - }, - function (intCallback) { - // Get the market information from shapeshift, which includes deposit limits, minimum deposits, rates, etc. - shapeshift.marketInfo(global.config.payout.shapeshiftPair, function (err, marketInfo) { - if (err) { - intCallback(err); - } else if (!marketInfo.hasOwnProperty("limit") || marketInfo.limit <= global.support.coinToDecimal(amount)) { - intCallback("Not enough coin in shapeshift to process at this time."); - } else if (!marketInfo.hasOwnProperty("min") || marketInfo.min >= global.support.coinToDecimal(amount)) { - intCallback("Not enough coin to hit the shapeshift minimum deposits."); - } else { - intCallback(null, marketInfo); - } - }); - }, - function (marketInfo, intCallback) { - // Validated there's enough coin. Time to make our dank txn. - // Return: - /* - { - "orderId": "cc49c556-e645-4c15-a943-d50a935274e4", - "sAddress": "46yzCCD3Mza9tRj7aqPSaxVbbePtuAeKzf8Ky2eRtcXGcEgCg1iTBio6N4sPmznfgGEUGDoBz5CLxZ2XPTyZu1yoCAG7zt6", - "deposit": "d8041668718e6e9d9d0fd335ee5ecd923e6fd074c41316d041cc18b779ade10e", - "depositType": "XMR", - "withdrawal": "1DbxcoCBSA9N7uZvkcvWxuLxSau9q9Pwiu", - "withdrawalType": "BTC", - "public": null, - "apiPubKey": "shapeshift", - "returnAddress": "46XWBqE1iwsVxSDP1qDrxhE1XvsZV6eALG5LwnoMdjbT4GPdy2bZTb99kagzxp2MMjUamTYZ4WgvZdFadvMimTjvR6Gv8hL", - "returnAddressType": "XMR" - } - Valid Statuses: - "received" - "complete" - "error" - "no_deposits" - Complete State Information: - { - "status": "complete", - "address": "d8041668718e6e9d9d0fd335ee5ecd923e6fd074c41316d041cc18b779ade10e", - "withdraw": "1DbxcoCBSA9N7uZvkcvWxuLxSau9q9Pwiu", - "incomingCoin": 3, - "incomingType": "XMR", - "outgoingCoin": "0.04186155", - "outgoingType": "BTC", - "transaction": "be9d97f6fc75262151f8f63e035c6ed638b9eb2a4e93fef43ea63124b045dbfb" - } - */ - shapeshift.shift(address, global.config.payout.shapeshiftPair, {returnAddress: global.config.pool.address}, function (err, returnData) { - if (err) { - intCallback(err); - } else { - global.mysql.query("INSERT INTO shapeshiftTxn (id, address, paymentID, depositType, withdrawl, withdrawlType, returnAddress, returnAddressType, txnStatus) VALUES (?,?,?,?,?,?,?,?,?)", - [returnData.orderId, returnData.sAddress, returnData.deposit, returnData.depositType, returnData.withdrawl, returnData.withdrawlType, returnData.returnAddress, returnData.returnAddressType, 'no_deposits']).then(function () { - intCallback(null, marketInfo, returnData); - }).catch(function (error) { - intCallback(error); - }); - } - }); - }, - function (marketInfo, shapeshiftTxnData, intCallback) { - // Make the payment to ShapeShift - let paymentDetails = { - destinations: [ - { - amount: amount, - address: shapeshiftTxnData.sAddress - } - ], - priority: global.config.payout.priority, - mixin: global.config.payout.mixIn, - payment_id: shapeshiftTxnData.deposit - }; - debug("Payment Details: " + JSON.stringify(paymentDetails)); - paymentQueue.push(paymentDetails, function (body) { - if (body.fee && body.fee > 10) { - intCallback(null, marketInfo, shapeshiftTxnData, body); - } else { - intCallback("Unknown error from the wallet."); - } - }); - }, - function (marketInfo, shapeshiftTxnData, body, intCallback) { - // body.tx_hash = XMR transaction hash. - // Need to add transaction. - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees, exchange_rate, exchange_name, exchange_txn_id) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", - [1, address, null, task.amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, global.support.decimalToCoin(marketInfo.minerFee), 1, global.support.decimalToCoin(marketInfo.rate), 'shapeshift', shapeshiftTxnData.orderId]).then(function (result) { - intCallback(null, result.insertId); - }).catch(function (error) { - intCallback(error); - }); - } - ], function (err, result) { - if (err) { - console.error("Error processing shapeshift txn: " + JSON.stringify(err)); - callback(true); - } else { - // Need to fill out this data pronto! - console.log("Processed ShapeShift transaction for: " + address + " Paid out: " + result + " payments in the db"); - callback(null, result); - } - }); -}, 2); +let is_full_stop = false; -let xmrToQueue = async.queue(function (task, callback) { - // http://xmrto-api.readthedocs.io/en/latest/introduction.html - // Documentation looks good! - // Amount needs to be shifted in as a non-completed value, as the wallet will only take non-complete values.. - let amount = task.amount - task.fee; - // Address is the destination address IN BTC. - let address = task.address; - // PaymentIDs are the paymentID's to flag as paid by this transaction. - // Should be a massive list of ID's so we can bulk-update them, by merging them with 's. - // Here we go! General process: Scan shapeshift for valid amounts of funds to xfer around. - // Once there's enough funds, then we active txn - // Do a wallet call to xfer. - // Setup a monitor on the transaction - async.waterfall([ - function (intCallback) { - // Verify if XMR.to is ready to get to work. - xmrAPIClient.get('order_parameter_query/', function (err, res, body) { - if (err) { - return intCallback(err); - } else if (body.error_msg) { - return intCallback(body.error_msg); - } else { - let amtOfBTC = ((amount / global.config.general.sigDivisor) * body.price).toPrecision(5); - console.log("Attempting to pay: " + address + " Amount: " + amtOfBTC + " BTC or " + amount / global.config.general.sigDivisor + " XMR"); - console.log("Response from XMR.to: " + JSON.stringify(body)); - if (body.lower_limit >= amtOfBTC) { - return intCallback("Not enough XMR to hit the minimum deposit"); - } else if (body.upper_limit <= amtOfBTC) { - return intCallback("Too much XMR to pay out to xmr.to"); - } else { - return intCallback(null, amtOfBTC); - } - } - }); - }, - function (btcValue, intCallback) { - // Validated there's enough coin. Time to make our dank txn. - // Return: - /* - { - "state": "TO_BE_CREATED", - "btc_amount": , - "btc_dest_address": "", - "uuid": "" - } - Valid Statuses: - "TO_BE_CREATED" - "UNPAID" - "UNDERPAID" - "PAID_UNCONFIRMED" - "PAID" - "BTC_SENT" - "TIMED_OUT" - "NOT_FOUND" - // Create, then immediately update with the new information w/ a status call. - */ - console.log("Amount of BTC to pay: " + btcValue); - xmrAPIClient.post('order_create/', { - btc_amount: btcValue, - btc_dest_address: address - }, function (err, res, body) { - if (err) { - return intCallback(err); - } else if (body.error_msg) { - return intCallback(body.error_msg); - } else { - return intCallback(null, body.uuid); - } - }); - }, - function (txnID, intCallback) { - // This function only exists because xmr.to is a pretty little fucking princess. - async.doUntil(function (xmrCallback) { - xmrAPIClient.post('order_status_query/', {uuid: txnID}, function (err, res, body) { - if (err) { - return intCallback(err); - } else if (body.error_msg) { - return intCallback(body.error_msg); - } else { - xmrCallback(null, body.state); - } - }); - }, - function (xmrCallback) { - return xmrCallback !== "TO_BE_CREATED"; - }, - function () { - intCallback(null, txnID); - }); - }, - function (txnID, intCallback) { - xmrAPIClient.post('order_status_query/', {uuid: txnID}, function (err, res, body) { - if (err) { - return intCallback(err); - } else if (body.error_msg) { - return intCallback(body.error_msg); - } else { - console.log(JSON.stringify(body)); - global.mysql.query("INSERT INTO xmrtoTxn (id, address, paymentID, depositType, withdrawl, withdrawlType, returnAddress, returnAddressType, txnStatus, amountDeposited, amountSent) VALUES (?,?,?,?,?,?,?,?,?,?,?)", - [txnID, body.xmr_receiving_address, body.xmr_required_payment_id, 'XMR', body.btc_dest_address, 'BTC', global.config.pool.address, 'XMR', body.state_str, global.support.decimalToCoin(body.xmr_amount_total), global.support.decimalToCoin(body.btc_amount)]).then(function () { - return intCallback(null, body, global.support.decimalToCoin(body.xmr_amount_total)); - }).catch(function (error) { - return intCallback(error); - }); - } - }); - }, - function (orderStatus, xmrDeposit, intCallback) { - // Make the payment to ShapeShift - let paymentDetails = { - destinations: [ - { - amount: xmrDeposit, - address: orderStatus.xmr_receiving_address - } - ], - priority: global.config.payout.priority, - mixin: global.config.payout.mixIn, - payment_id: orderStatus.xmr_required_payment_id - }; - debug("Payment Details: " + JSON.stringify(paymentDetails)); - paymentQueue.push(paymentDetails, function (body) { - if (body.fee && body.fee > 10) { - return intCallback(null, orderStatus, body); - } else { - return intCallback("Unknown error from the wallet."); - } - }); - }, - function (orderStatus, body, intCallback) { - // body.tx_hash = XMR transaction hash. - // Need to add transaction. - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees, exchange_rate, exchange_name, exchange_txn_id) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)", - [1, address, null, global.support.decimalToCoin(orderStatus.xmr_amount_total), body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, 1, global.support.decimalToCoin(orderStatus.xmr_price_btc), 'xmrto', orderStatus.uuid]).then(function (result) { - return intCallback(null, result.insertId); - }).catch(function (error) { - return intCallback(error); - }); - } - ], function (err, result) { - if (err) { - console.error("Error processing XMRTo txn: " + JSON.stringify(err)); - return callback("Error!"); - } else { - // Need to fill out this data pronto! - console.log("Processed XMRTo transaction for: " + address + " Paid out: " + result + " payments in the db"); - return callback(null, result); - } - }); -}, 2); +function full_stop(err) { + is_full_stop = true; + console.error("Issue making payments: " + JSON.stringify(err)); + console.error("Will not make more payments until the payment daemon is restarted!"); + //toAddress, subject, body + global.support.sendEmail(global.config.general.adminEmail, "Payment daemon unable to make payment", + "Hello,\r\nThe payment daemon has hit an issue making a payment: " + JSON.stringify(err) + + ". Please investigate and restart the payment daemon as appropriate"); +} let paymentQueue = async.queue(function (paymentDetails, callback) { + if (is_full_stop) { + debug("Dropping all pending payments"); + return; + } + /* support JSON URI: http://10.0.0.2:28082/json_rpc Args: {"id":"0","jsonrpc":"2.0","method":"transfer","params":{"destinations":[{"amount":68130252045355,"address":"A2MSrn49ziBPJBh8ZNEhhbfyLMou6mao4C1F5TLGUatmUnCxZArDYkcbAnVkVEopWVeak2rKDrmc8JpoS7n5dvfN9YDPBTG"}],"mixin":4,"payment_id":"7e52c5266de9fede7fb3abc0cd88f937b38b51426f7b34ff99729d28ce4e1142"}} +1ms payments Payment made: {"id":"0","jsonrpc":"2.0","result":{"fee":40199391255,"tx_hash":"c418708643f72635edf522490bfb2cae9d42a6dc1df30dcde844862dfd88f5b3","tx_key":""}} +2s */ - if (paymentTimer !== null){ - clearInterval(paymentTimer); - paymentTimer = null; - } - debug("Making payment based on: " + JSON.stringify(paymentDetails)); - let transferFunc = 'transfer'; - global.support.rpcWallet(transferFunc, paymentDetails, function (body) { - debug("Payment made: " + JSON.stringify(body)); - if (body.hasOwnProperty('error')) { - if (body.error.message === "not enough money"){ - console.error("Issue making payments, not enough money, will try later"); - if(!extraPaymentRound){ - setTimeout(function(){ - makePayments(); - }, global.config.payout.timerRetry * 60 * 1000); - } - extraPaymentRound = true; - return callback(false); - } else { - console.error("Issue making payments" + JSON.stringify(body.error)); - console.error("Will not make more payments until the payment daemon is restarted!"); - //toAddress, subject, body - global.support.sendEmail(global.config.general.adminEmail, "Payment daemon unable to make payment", - "Hello,\r\nThe payment daemon has hit an issue making a payment: " + JSON.stringify(body.error) + - ". Please investigate and restart the payment daemon as appropriate"); + + debug("Trying to make payment based on: " + JSON.stringify(paymentDetails)); + + function getbalance() { + global.support.rpcWallet("getbalance", paymentDetails, function (body) { + if (body.hasOwnProperty('error') || !body.hasOwnProperty('result') || typeof(body.result) === 'undefined' || !body.result.hasOwnProperty('unlocked_balance') || typeof(body.result.unlocked_balance) !== "number") { + console.error("Can't getbalance: " + JSON.stringify(body.error)); + setTimeout(getbalance, 60*1000); return; } - } - if (paymentDetails.hasOwnProperty('payment_id')) { - console.log("Payment made to " + paymentDetails.destinations[0].address + " with PaymentID: " + paymentDetails.payment_id + " For: " + global.support.coinToDecimal(paymentDetails.destinations[0].amount) + " XMR with a " + global.support.coinToDecimal(body.result.fee) + " XMR Mining Fee"); - return callback(body.result); - } else { - if (transferFunc === 'transfer') { - console.log("Payment made out to multiple people, total fee: " + global.support.coinToDecimal(body.result.fee) + " XMR"); + if (body.result.unlocked_balance === 0) { + console.log("Waiting for balance to unlock after previous payment"); + setTimeout(getbalance, 5*60*1000); + return; } - let intCount = 0; - paymentDetails.destinations.forEach(function (details) { - console.log("Payment made to: " + details.address + " For: " + global.support.coinToDecimal(details.amount) + " XMR"); - intCount += 1; - if (intCount === paymentDetails.destinations.length) { - return callback(body.result); - } - }); - } - }); -}, 1); - -paymentQueue.drain = function(){ - extraPaymentRound = false; - if (global.config.payout.timer > 35791){ - console.error("Payout timer is too high. Please use a value under 35791 to avoid overflows."); - } else { - paymentTimer = setInterval(makePayments, global.config.payout.timer * 60 * 1000); - } - global.database.setCache('lastPaymentCycle', Math.floor(Date.now()/1000)); -}; + console.log("Current wallet balance is " + global.support.coinToDecimal(body.result.balance) + " with " + global.support.coinToDecimal(body.result.unlocked_balance) + " unlocked balance"); -function updateShapeshiftCompletion() { - global.mysql.query("SELECT * FROM shapeshiftTxn WHERE txnStatus NOT IN ('complete', 'error')").then(function (rows) { - rows.forEach(function (row) { - shapeshift.status(row.paymentID, function (err, status, returnData) { - if (err) { - return; - } - global.mysql.query("UPDATE shapeshiftTxn SET txnStatus = ? WHERE id = ?", [status, row.id]).then(function () { - if (status === 'complete') { - global.mysql.query("UPDATE shapeshiftTxn SET amountDeposited = ?, amountSent = ?, transactionHash = ? WHERE id = ?", - [global.support.decimalToCoin(returnData.incomingCoin), global.support.bitcoinDecimalToCoin(returnData.outgoingCoin), returnData.transaction, row.id]).then(function () { - global.mysql.query("UPDATE transactions SET confirmed = 1, confirmed_time = now(), btc_amt = ? WHERE exchange_txn_id = ?", [global.support.bitcoinDecimalToCoin(returnData.outgoingCoin), row.id]); - }); - } else if (status === 'error') { - // Failed txn. Need to rollback and delete all related data. Here we go! - global.mysql.query("DELETE FROM shapeshiftTxn WHERE id = ?", [row.id]); - global.mysql.query("SELECT id, xmr_amt, address FROM transactions WHERE exchange_txn_id = ?", [row.id]).then(function (rows) { - global.mysql.query("DELETE FROM transactions WHERE id = ?", [rows[0].id]); - global.mysql.query("DELETE payments WHERE transaction_id = ?", [rows[0].id]); - global.mysql.query("UPDATE balance SET amount = amount+? WHERE payment_address = ? limit 1", [rows[0].xmr_amt, rows[0].address]); - }); - console.error("Failed transaction from ShapeShift " + JSON.stringify(returnData)); + let transferFunc = 'transfer'; + paymentDetails.get_tx_key = true; + global.support.rpcWallet(transferFunc, paymentDetails, function (body) { + debug("Payment made: " + JSON.stringify(body)); + if (body.hasOwnProperty('error') || !body.hasOwnProperty('result')) { + if ( typeof(body.error) !== 'undefined' && + body.error.hasOwnProperty('message') && + ( body.error.message === "not enough money" || + body.error.message === "not enough unlocked money" || + body.error.message === "transaction was rejected by daemon" + ) + ) { + console.error("Issue making payments, not enough money, will try later"); + setTimeout(getbalance, 10*60*1000); + } else { + full_stop(body.error); } - }); - }); - }); - }); -} - -function updateXMRToCompletion() { - global.mysql.query("SELECT * FROM xmrtoTxn WHERE txnStatus NOT IN ('PAID', 'TIMED_OUT', 'NOT_FOUND', 'BTC_SENT')").then(function (rows) { - rows.forEach(function (row) { - xmrAPIClient.post('order_status_query/', {uuid: row.id}, function (err, res, body) { - if (err) { - console.log("Error in getting order status: " + JSON.stringify(err)); return; } - if (body.error_msg) { - console.log("Error in getting order status: " + body.error_msg); - return; - } - global.mysql.query("UPDATE xmrtoTxn SET txnStatus = ? WHERE id = ?", [body.state, row.id]).then(function () { - if (body.status === 'BTC_SENT') { - global.mysql.query("UPDATE xmrtoTxn SET transactionHash = ? WHERE id = ?", [body.btc_transaction_id, row.id]).then(function () { - global.mysql.query("UPDATE transactions SET confirmed = 1, confirmed_time = now(), btc_amt = ? WHERE exchange_txn_id = ?", [global.support.bitcoinDecimalToCoin(body.btc_amount), row.id]); - }); - } else if (body.status === 'TIMED_OUT' || body.status === 'NOT_FOUND') { - global.mysql.query("DELETE FROM xmrtoTxn WHERE id = ?", [row.id]); - global.mysql.query("SELECT id, xmr_amt, address FROM transactions WHERE exchange_txn_id = ?", [row.id]).then(function (rows) { - global.mysql.query("DELETE FROM transactions WHERE id = ?", [rows[0].id]); - global.mysql.query("DELETE payments WHERE transaction_id = ?", [rows[0].id]); - global.mysql.query("UPDATE balance SET amount = amount+? WHERE payment_address = ? limit 1", [rows[0].xmr_amt, rows[0].address]); - }); - console.error("Failed transaction from XMRto " + JSON.stringify(body)); - } - }); + callback(body.result); }); }); - }); -} + }; -function determineBestExchange() { - async.waterfall([ - function (callback) { - // Verify if the coin is active in ShapeShift first. - shapeshift.coins(function (err, coinData) { - if (err) { - return callback(err); - } else if (!coinData.hasOwnProperty(global.config.general.coinCode) || coinData[global.config.general.coinCode].status !== "available") { - return callback("Coin " + global.config.general.coinCode + " Is not available at this time on shapeshift."); - } else { - return callback(null); - } - }); - }, - function (callback) { - // Get the market information from shapeshift, which includes deposit limits, minimum deposits, rates, etc. - shapeshift.marketInfo(global.config.payout.shapeshiftPair, function (err, marketInfo) { - if (err) { - return callback(err); - } else if (!marketInfo.hasOwnProperty("rate")) { - return callback("Shapeshift did not return the rate."); - } else { - return callback(null, global.support.bitcoinDecimalToCoin(marketInfo.rate)); - } - }); - }, - function (ssValue, callback) { - xmrAPIClient.get('order_parameter_query/', function (err, res, body) { - console.log("XMR.to pricing body: " + JSON.stringify(body)); - if (err) { - return callback(err); - } else if (body.error_msg) { - return callback(body.error_msg); - } else { - return callback(null, ssValue, global.support.bitcoinDecimalToCoin(body.price)); - } - }); - } - ], function (err, ssValue, xmrToValue) { - if (err) { - return console.error("Error processing exchange value: " + JSON.stringify(err)); - } - debug("ShapeShift Value: " + global.support.bitcoinCoinToDecimal(ssValue) + " XMR.to Value: " + global.support.bitcoinCoinToDecimal(xmrToValue)); - if (ssValue >= xmrToValue) { - console.log("ShapeShift is the better BTC exchange, current rate: " + global.support.bitcoinCoinToDecimal(ssValue)); - bestExchange = 'shapeshift'; - global.mysql.query("UPDATE config SET item_value = 'shapeshift' where item='bestExchange'"); - global.mysql.query("UPDATE config SET item_value = ? where item='exchangeRate'", [ssValue]); - } else { - console.log("XMR.to is the better BTC exchange, current rate: " + global.support.bitcoinCoinToDecimal(xmrToValue)); - bestExchange = 'xmrto'; - global.mysql.query("UPDATE config SET item_value = 'xmrto' where item='bestExchange'"); - global.mysql.query("UPDATE config SET item_value = ? where item='exchangeRate'", [xmrToValue]); - } - }); -} + getbalance(); + +}, 1); + +paymentQueue.drain(function(){ + console.log("Payment queue drained"); + global.database.setCache('lastPaymentCycle', Math.floor(Date.now()/1000)); +}); -function Payee(amount, address, paymentID, bitcoin) { +function Payee(amount, address, paymentID) { this.amount = amount; this.address = address; this.paymentID = paymentID; - this.bitcoin = bitcoin; this.blockID = 0; this.poolType = ''; this.transactionID = 0; @@ -510,20 +118,41 @@ function Payee(amount, address, paymentID, bitcoin) { }; let identifier = this.id; let amount = this.amount; + let fee = this.fee; let address = this.address; let paymentID = this.paymentID; let payee = this; debug("Payment Details: " + JSON.stringify(paymentDetails)); paymentQueue.push(paymentDetails, function (body) { if (body.fee && body.fee > 10) { - debug("Successful payment sent to: " + identifier); - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [0, address, paymentID, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, 1]).then(function (result) { + console.log("[*] Successful payment to " + identifier + " of " + global.support.coinToDecimal(amount) + " XMR (fee " + global.support.coinToDecimal(fee) + " - " + global.support.coinToDecimal(body.fee) + " = " + global.support.coinToDecimal(fee - body.fee) + ") with tx_hash " + body.tx_hash.match(hexChars)[0] + " and tx_key " + body.tx_key); + global.mysql.query("INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?)", + [address, paymentID, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, 1]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do: INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES ('" + + address + "', '" + paymentID + "', " + amount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", 1)" + ); + payee.transactionID = 0; + payee.manualPaymentShow(); + full_stop(result); + return; + } payee.transactionID = result.insertId; + payee.tx_hash = body.tx_hash.match(hexChars)[0]; + payee.tx_key = body.tx_key; payee.trackPayment(); + }).catch(function (error) { + console.error("SQL query failed: " + error); + console.error("Can't do: INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES ('" + + address + "', '" + paymentID + "', " + amount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", 1)" + ); + payee.transactionID = 0; + payee.manualPaymentShow(); + full_stop(result); + return; }); } else { - console.error("Unknown error from the wallet."); + console.error("Unknown error from the wallet: " + JSON.stringify(body)); } }); }; @@ -541,70 +170,141 @@ function Payee(amount, address, paymentID, bitcoin) { }; let identifier = this.id; let amount = this.amount; + let fee = this.fee; let address = this.address; let payee = this; debug("Payment Details: " + JSON.stringify(paymentDetails)); paymentQueue.push(paymentDetails, function (body) { if (body.fee && body.fee > 10) { - debug("Successful payment sent to: " + identifier); - global.mysql.query("INSERT INTO transactions (bitcoin, address, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?)", - [0, address, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, 1]).then(function (result) { + console.log("[*] Successful payment to " + identifier + " of " + global.support.coinToDecimal(amount) + " XMR (fee " + global.support.coinToDecimal(fee) + " - " + global.support.coinToDecimal(body.fee) + " = " + global.support.coinToDecimal(fee - body.fee) + ") with tx_hash " + body.tx_hash.match(hexChars)[0] + " and tx_key " + body.tx_key); + global.mysql.query("INSERT INTO transactions (address, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?)", + [address, amount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, 1]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do: INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES ('" + + address + "', " + amount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", 1)" + ); + payee.transactionID = 0; + payee.manualPaymentShow(); + full_stop(result); + return; + } payee.transactionID = result.insertId; + payee.tx_hash = body.tx_hash.match(hexChars)[0]; + payee.tx_key = body.tx_key; payee.trackPayment(); + }).catch(function (error) { + console.error("SQL query failed: " + error); + console.error("Can't do: INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES ('" + + address + "', " + amount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", 1)" + ); + payee.transactionID = 0; + payee.manualPaymentShow(); + full_stop(result); + return; }); } else { - console.error("Unknown error from the wallet."); + console.error("Unknown error from the wallet: " + JSON.stringify(body)); } }); }; - this.makeBitcoinPayment = function () { - let functionalData = {address: this.address, amount: this.amount, fee: this.fee}; - let payee = this; - if (bestExchange === 'xmrto') { - xmrToQueue.push(functionalData, function (err, transactionID) { - if (err) { - return console.error("Error processing payment for " + functionalData.address); - } - payee.transactionID = transactionID; - payee.trackPayment(); - }); - } else { - shapeshiftQueue.push(functionalData, function (err, transactionID) { - if (err) { - return console.error("Error processing payment for " + functionalData.address); - } - payee.transactionID = transactionID; - payee.trackPayment(); - }); - } + this.manualPaymentShow = function () { + console.error("Manual payment update:"); + console.error(" UPDATE balance SET amount = amount - " + this.amount + " WHERE id = " + this.sqlID + ";"); + console.error(" INSERT INTO payments (unlocked_time, paid_time, pool_type, payment_address, transaction_id, amount, payment_id, transfer_fee) VALUES (now(), now(), " + + this.poolType + ", " + this.address + ", " + this.transactionID + ", " + (this.amount - this.fee) + ", " + this.paymentID + ", " + this.fee + ");" + ); }; this.trackPayment = function () { - global.mysql.query("UPDATE balance SET amount = amount - ? WHERE id = ?", [this.amount, this.sqlID]); - global.mysql.query("INSERT INTO payments (unlocked_time, paid_time, pool_type, payment_address, transaction_id, bitcoin, amount, payment_id, transfer_fee)" + - " VALUES (now(), now(), ?, ?, ?, ?, ?, ?, ?)", [this.poolType, this.address, this.transactionID, this.bitcoin, this.amount - this.fee, this.paymentID, this.fee]); + global.mysql.query("UPDATE balance SET amount = amount - ? WHERE id = ?", [this.amount, this.sqlID]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do SQL balance update"); + this.manualPaymentShow(); + full_stop(result); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + console.error("Can't do SQL balance update"); + this.manualPaymentShow(); + full_stop(result); + }); + global.mysql.query("INSERT INTO payments (unlocked_time, paid_time, pool_type, payment_address, transaction_id, amount, payment_id, transfer_fee)" + + " VALUES (now(), now(), ?, ?, ?, ?, ?, ?)", [this.poolType, this.address, this.transactionID, this.amount - this.fee, this.paymentID, this.fee]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do SQL payments update"); + this.manualPaymentShow(); + full_stop(result); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + console.error("Can't do SQL payments update"); + this.manualPaymentShow(); + full_stop(result); + }); + + let payee = this; + + global.mysql.query("SELECT email FROM users WHERE username = ? AND enable_email IS true limit 1", [payee.id]).then(function (rows) { + if (rows.length === 0) return; + // toAddress, subject, body + let emailData = { + address: payee.address, + address2: payee.id, + payment_amount: global.support.coinToDecimal(payee.amount - payee.fee), + amount: global.support.coinToDecimal(payee.amount), + fee: global.support.coinToDecimal(payee.fee), + tx_hash: payee.tx_hash, + tx_key: payee.tx_key + }; + global.support.sendEmail(rows[0].email, + sprintf("Your %(payment_amount)s XMR payment was just performed", emailData), + sprintf( + "Your payment of %(payment_amount)s XMR (with tx fee %(fee)s XMR) to %(address2)s wallet was just performed and total due was decreased by %(amount)s XMR.\n" + + (payee.tx_hash && payee.tx_key ? + "Your payment tx_hash (tx_id) is %(tx_hash)s and tx_key is %(tx_key)s (can be used to verify payment)\n" + + "Here is link to verify that this payment was made: https://xmrchain.net/prove/%(tx_hash)s/%(address)s/%(tx_key)s\n" + + "You can also check that in your command line (cli) wallet using \"check_tx_key %(tx_hash)s %(tx_key)s %(address)s\" command " + + "(see https://getmonero.org/resources/user-guides/prove-payment.html for more details)\n" + : "" + ), + emailData + ), + payee.id + ); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); }; } function makePayments() { + if (is_full_stop) { + debug("Dropping all new payment creation"); + return; + } + if (paymentQueue.idle() === false) { + debug("Payment queue is not empty so dropping all new payment creation"); + return; + } + + console.log("Starting makePayments"); global.mysql.query("SELECT * FROM balance WHERE amount >= ?", [global.support.decimalToCoin(global.config.payout.walletMin)]).then(function (rows) { console.log("Loaded all payees into the system for processing"); let paymentDestinations = []; let totalAmount = 0; - let roundCount = 0; - let payeeList = []; let payeeObjects = {}; - rows.forEach(function (row) { - debug("Starting round for: " + JSON.stringify(row)); - let payee = new Payee(row.amount, row.payment_address, row.payment_id, row.bitcoin); - payeeObjects[row.payment_address] = payee; + async.eachSeries(rows, function(row, next) { + //debug("Starting round for: " + JSON.stringify(row)); + if ((row.payment_address + (row.payment_id ? ('.' + row.payment_id) : '')) in payeeObjects) return next(); // avoid doing payment for different pool types at the same time + let payee = new Payee(row.amount, row.payment_address, row.payment_id); global.mysql.query("SELECT payout_threshold FROM users WHERE username = ?", [payee.id]).then(function (userRow) { - roundCount += 1; - let threshold = 0; - if (userRow.length !== 0) { + let threshold = global.support.decimalToCoin(global.config.payout.defaultPay); + let custom_threshold = false; + if (userRow.length !== 0 && userRow[0].payout_threshold != 0) { threshold = userRow[0].payout_threshold; + custom_threshold = true; } payee.poolType = row.pool_type; payee.sqlID = row.id; @@ -619,78 +319,113 @@ function makePayments() { if (remainder !== 0) { payee.amount -= remainder; } - if (payee.amount > threshold) { + if (payee.amount >= threshold) { payee.setFeeAmount(); - if (payee.bitcoin === 0 && payee.paymentID === null && payee.amount !== 0 && payee.amount > 0 && payee.address.length !== 106) { - debug("Adding " + payee.id + " to the list of people to pay (OG Address). Payee balance: " + global.support.coinToDecimal(payee.amount)); - paymentDestinations.push({amount: payee.amount - payee.fee, address: payee.address}); - totalAmount += payee.amount; - payeeList.push(payee); - } else if (payee.bitcoin === 0 && payee.paymentID === null && payee.amount !== 0 && payee.amount > 0 && payee.address.length === 106 && (payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && threshold !== 0))) { - // Special code to handle integrated payment addresses. What a pain in the rear. - // These are exchange addresses though, so they need to hit the exchange payout amount. - debug("Adding " + payee.id + " to the list of people to pay (Integrated Address). Payee balance: " + global.support.coinToDecimal(payee.amount)); - payee.makePaymentAsIntegrated(); - } else if ((payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && threshold !== 0)) && payee.bitcoin === 0) { - debug("Adding " + payee.id + " to the list of people to pay (Payment ID Address). Payee balance: " + global.support.coinToDecimal(payee.amount)); - payee.makePaymentWithID(); - } else if ((payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || (payee.amount > threshold && threshold !== 0)) && payee.bitcoin === 1) { - debug("Adding " + payee.id + " to the list of people to pay (Bitcoin Payout). Payee balance: " + global.support.coinToDecimal(payee.amount)); - payee.makeBitcoinPayment(); + if (payee.paymentID === null && payee.amount !== 0 && payee.amount > 0) { + if (payee.address.length !== 106 ) { + payeeObjects[payee.id] = payee; + console.log("[++] " + payee.id + " miner to bulk payment. Amount: " + global.support.coinToDecimal(payee.amount)); + paymentDestinations.push({amount: payee.amount - payee.fee, address: payee.address}); + totalAmount += payee.amount; + } else if ( payee.amount >= global.support.decimalToCoin(global.config.payout.exchangeMin) || + ( payee.amount > threshold && custom_threshold ) + ) { + // Special code to handle integrated payment addresses. What a pain in the rear. + // These are exchange addresses though, so they need to hit the exchange payout amount. + payeeObjects[payee.id] = payee; + console.log("[+] " + payee.id + " as separate payment to integrated address. Amount: " + global.support.coinToDecimal(payee.amount)); + payee.makePaymentAsIntegrated(); + } } } - debug("Went: " + roundCount + " With: " + paymentDestinations.length + " Possible destinations and: " + rows.length + " Rows"); - if (roundCount === rows.length && paymentDestinations.length > 0) { - while (paymentDestinations.length > 0) { - let paymentDetails = { - destinations: paymentDestinations.splice(0, global.config.payout.maxPaymentTxns), - priority: global.config.payout.priority, - mixin: global.config.payout.mixIn - }; - console.log("Paying out: " + paymentDetails.destinations.length + " people"); - paymentQueue.push(paymentDetails, function (body) { //jshint ignore:line - // This is the only section that could potentially contain multiple txns. Lets do this safely eh? - if (body.fee && body.fee > 10) { - debug("Made it to the SQL insert for transactions"); - let totalAmount = 0; + return next(); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); + }, function() { + while (paymentDestinations.length > 0) { + let paymentDetails = { + destinations: paymentDestinations.splice(0, global.config.payout.maxPaymentTxns), + priority: global.config.payout.priority, + mixin: global.config.payout.mixIn + }; + console.log("Adding payment for " + paymentDetails.destinations.length + " miners"); + paymentQueue.unshift(paymentDetails, function (body) { //jshint ignore:line + // This is the only section that could potentially contain multiple txns. Lets do this safely eh? + if (body.fee && body.fee > 10) { + let totalAmount = 0; + let totalFee = 0; + paymentDetails.destinations.forEach(function (payeeItem) { + totalAmount += payeeObjects[payeeItem.address].amount; + totalFee += payeeObjects[payeeItem.address].fee; + console.log("[**] Successful payment to " + payeeItem.address + " for " + global.support.coinToDecimal(payeeObjects[payeeItem.address].amount) + " XMR (fee " + global.support.coinToDecimal(payeeObjects[payeeItem.address].fee) + ")"); + }); + console.log("[*] Successful payment to multiple miners of " + global.support.coinToDecimal(totalAmount) + " XMR (fee " + global.support.coinToDecimal(totalFee) + " - " + global.support.coinToDecimal(body.fee) + " = " + global.support.coinToDecimal(totalFee - body.fee) + ") with tx_hash " + body.tx_hash.match(hexChars)[0] + " and tx_key " + body.tx_key); + global.mysql.query("INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?)", + [null, null, totalAmount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, paymentDetails.destinations.length]).then(function (result) { + if (!result.hasOwnProperty("affectedRows") || result.affectedRows != 1) { + console.error("Can't do: INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (null, null, " + + totalAmount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", " + paymentDetails.destinations.length + ")" + ); paymentDetails.destinations.forEach(function (payeeItem) { - totalAmount += payeeObjects[payeeItem.address].amount; - totalAmount += payeeObjects[payeeItem.address].fee; + let payee = payeeObjects[payeeItem.address]; + payee.transactionID = 0; + payee.manualPaymentShow(); }); - global.mysql.query("INSERT INTO transactions (bitcoin, address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (?, ?, ?, ?, ?, ?, ?, ?)", - [0, null, null, totalAmount, body.tx_hash.match(hexChars)[0], global.config.payout.mixIn, body.fee, paymentDetails.destinations.length]).then(function (result) { - paymentDetails.destinations.forEach(function (payeeItem) { - payee = payeeObjects[payeeItem.address]; - payee.transactionID = result.insertId; - payee.trackPayment(); - }); - }); - } else { - console.error("Unknown error from the wallet."); + full_stop(result); + return; } + paymentDetails.destinations.forEach(function (payeeItem) { + let payee = payeeObjects[payeeItem.address]; + payee.transactionID = result.insertId; + payee.tx_hash = body.tx_hash.match(hexChars)[0]; + payee.tx_key = body.tx_key; + payee.trackPayment(); + }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + console.error("Can't do: INSERT INTO transactions (address, payment_id, xmr_amt, transaction_hash, mixin, fees, payees) VALUES (null, null, " + + totalAmount + ", '" + body.tx_hash.match(hexChars)[0] + "', " + global.config.payout.mixIn + ", " + body.fee + ", " + paymentDetails.destinations.length + ")" + ); + paymentDetails.destinations.forEach(function (payeeItem) { + let payee = payeeObjects[payeeItem.address]; + payee.transactionID = 0; + payee.manualPaymentShow(); + }); + full_stop(result); + return; }); + } else { + console.error("Unknown error from the wallet: " + JSON.stringify(body)); } - } - }); + }); + } + debug("Finished processing payments for now"); }); + }).catch(function (error) { + console.error("SQL query failed: " + error); }); + debug("Finished makePayments"); } function init() { - global.support.rpcWallet("store", [], function () { - }); - if (global.config.allowBitcoin) { - determineBestExchange(); - setInterval(updateXMRToCompletion, 90000); - setInterval(updateShapeshiftCompletion, 90000); - setInterval(determineBestExchange, 60000); - } + global.support.rpcWallet("store", [], function () {}); setInterval(function () { - global.support.rpcWallet("store", [], function () { - }); - }, 60000); - console.log("Setting the payment timer to: " + global.config.payout.timer + " minutes with a: " + global.config.payout.timerRetry + " minute delay if the wallet is out of money"); + global.support.rpcWallet("store", [], function () {}); + }, 60*1000); + + setInterval(function () { + console.log("Payment queue lengths: payment (" + (paymentQueue.running() + paymentQueue.length()) + ")"); + }, 10*60*1000); + makePayments(); + + console.log("Setting the payment timer to: " + global.config.payout.timer + " minutes"); + setInterval(makePayments, global.config.payout.timer * 60 * 1000); } -init(); \ No newline at end of file +if (global.config.payout.timer > 35791) { + console.error("Payout timer is too high. Please use a value under 35791 to avoid overflows."); +} else { + init(); +} diff --git a/lib/pool.js b/lib/pool.js index c61c911e..4f0ca6b4 100644 --- a/lib/pool.js +++ b/lib/pool.js @@ -1,6 +1,5 @@ "use strict"; const debug = require('debug')('pool'); -const uuidV4 = require('uuid/v4'); const crypto = require('crypto'); const bignum = require('bignum'); const cluster = require('cluster'); @@ -9,65 +8,133 @@ const async = require('async'); const net = require('net'); const tls = require('tls'); const fs = require('fs'); +const child_process = require('child_process'); + +//const httpResponse = ' 200 OK\nContent-Type: text/plain\nContent-Length: 18\n\nMining Pool Online'; +const nonceCheck32 = new RegExp("^[0-9a-f]{8}$"); +const nonceCheck64 = new RegExp("^[0-9a-f]{16}$"); +const hashCheck32 = new RegExp("^[0-9a-f]{64}$"); +const hexMatch = new RegExp("^(?:[0-9a-f][0-9a-f])+$"); +const localhostCheck = new RegExp(/127\.0\.0\.1$/); +const baseDiff = global.coinFuncs.baseDiff(); +const baseRavenDiff = global.coinFuncs.baseRavenDiff(); + +const BLOCK_NOTIFY_PORT = 2223; +const DAEMON_POLL_MS = 500; + +let decId = 0; +function get_new_id() { + if (++decId > 999999999999999) decId = 0; + return decId.toString(10); +}; + +function pad_hex(str, bytes) { + const bytes2 = bytes * 2; + return ("00".repeat(bytes) + str.substr(0, bytes2)).substr(-bytes2); +} + +let ethJobId = 0; + +function get_new_eth_job_id() { + if (++ethJobId > 0xFFFF) ethJobId = 0; + return pad_hex(ethJobId.toString(16), 2); +}; + +let uniqueWorkerId; +let uniqueWorkerIdBits; +let freeEthExtranonces = []; + +function get_new_eth_extranonce_id() { + if (!freeEthExtranonces.length) { + const err_str = threadName + "Pool server " + global.config.hostname + " has overlow extranonce of " + (16 - uniqueWorkerIdBits) + " bits"; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Pool node has extranonce overflow", err_str); + return null; + } + return freeEthExtranonces.pop(); +}; + +function eth_extranonce(id) { + return id === null ? null : pad_hex(((id << uniqueWorkerIdBits) + uniqueWorkerId).toString(16), 2); +}; + +let bannedTmpIPs = {}; // ip banned for short time +let bannedTmpWallets = {}; // wallets banned for short time +let bannedBigTmpWallets = {}; // wallets banned for a long time +let bannedAddresses = {}; // forever banned wallets +let notifyAddresses = {}; // wallet notifications + +let activeMiners = new Map(); + +let lastBlockHash = {}; // coin key +let lastBlockTime = {}; // coin key +let lastBlockKeepTime = {}; // coin key +let lastBlockReward = {}; // coin key +let activeBlockTemplates = {}; // coin key +let pastBlockTemplates = {}; // coin key -> global.support.circularBuffer -> activeBlockTemplates + +let newCoinHashFactor = {}; // coin key, current individual coin hash factor, set in updateCoinHashFactor +let lastCoinHashFactor = {}; // coin key, last set individual coin hash factor, set in setNewCoinHashFactor +let lastCoinHashFactorMM = {}; // coin key, current individual coin hash factor that includes merged mining factor, set in setNewCoinHashFactor + +let lastBlockFixTime = {}; // time when blocks were checked to be in line with other nodes or when fix_daemon_sh was attempted +let lastBlockFixCount = {}; // number of times fix_daemon_sh was run -let nonceCheck = new RegExp("^[0-9a-f]{8}$"); -let bannedIPs = []; -let bannedAddresses = []; -let baseDiff = global.coinFuncs.baseDiff(); -let pastBlockTemplates = global.support.circularBuffer(4); -let activeMiners = []; -let activeBlockTemplate; -let workerList = []; -let httpResponse = ' 200 OK\nContent-Type: text/plain\nContent-Length: 18\n\nMining Pool Online'; let threadName; let minerCount = []; -let BlockTemplate = global.coinFuncs.BlockTemplate; -let hexMatch = new RegExp("^[0-9a-f]+$"); -let totalShares = 0, trustedShares = 0, normalShares = 0, invalidShares = 0; +let totalShares = 0, trustedShares = 0, normalShares = 0, invalidShares = 0, outdatedShares = 0, throttledShares = 0; +// wallet -> { connectTime, count (miner), hashes, last_ver_shares } +// this is need to thottle down some high share count miners +let minerWallets = {}; Buffer.prototype.toByteArray = function () { return Array.prototype.slice.call(this, 0); }; - if (cluster.isMaster) { threadName = "(Master) "; setInterval(function () { - console.log(`Processed ${trustedShares}/${normalShares}/${invalidShares}/${totalShares} Trusted/Validated/Invalid/Total shares in the last 30 seconds`); - totalShares = 0; - trustedShares = 0; - normalShares = 0; - invalidShares = 0; - }, 30000); + let trustedSharesPercent = (totalShares ? trustedShares / totalShares * 100 : 0).toFixed(2); + let normalSharesPercent = (totalShares ? normalShares / totalShares * 100 : 0).toFixed(2); + let invalidSharesPercent = (totalShares ? invalidShares / totalShares * 100 : 0).toFixed(2); + let outdatedSharesPercent = (totalShares ? outdatedShares / totalShares * 100 : 0).toFixed(2); + let throttledSharesPercent = (totalShares ? throttledShares / totalShares * 100 : 0).toFixed(2); + console.log(`>>> Trusted=${trustedShares}(${trustedSharesPercent}%) / Validated=${normalShares}(${normalSharesPercent}%) / Invalid=${invalidShares}(${invalidSharesPercent}%) / Outdated=${outdatedShares}(${outdatedSharesPercent}%) / Throttled=${throttledShares}(${throttledSharesPercent}%) / Total=${totalShares} shares`); + totalShares = 0; + trustedShares = 0; + normalShares = 0; + invalidShares = 0; + outdatedShares = 0; + throttledShares = 0; + }, 30*1000); } else { - threadName = "(Worker " + cluster.worker.id + " - " + process.pid + ") "; + threadName = "(Worker " + process.env['WORKER_ID'] + " - " + process.pid + ") "; + // reset last verified share counters every global.config.pool.minerThrottleShareWindow seconds + setInterval(function () { + for (let wallet in minerWallets) { + minerWallets[wallet].last_ver_shares = 0; + } + }, global.config.pool.minerThrottleShareWindow*1000); } global.database.thread_id = threadName; +const COINS = global.coinFuncs.getCOINS(); + function registerPool() { - global.mysql.query("SELECT * FROM pools WHERE id = ?", [global.config.pool_id]).then(function (rows) { - rows.forEach(function (row) { - if (row.ip !== global.config.bind_ip) { - console.error("Pool ID in use already for a different IP. Update MySQL or change pool ID."); - process.exit(1); + global.mysql.query("INSERT INTO pools (id, ip, last_checkin, active, hostname) VALUES (?, ?, now(), ?, ?) ON DUPLICATE KEY UPDATE last_checkin=now(), active=?", + [global.config.pool_id, global.config.bind_ip, true, global.config.hostname, true]); + global.mysql.query("DELETE FROM ports WHERE pool_id = ?", [global.config.pool_id]).then(function () { + global.config.ports.forEach(function (port) { + if ('ssl' in port && port.ssl === true) { + global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 1)", + [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); + } else { + global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 0)", + [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); } }); - }).then(function () { - global.mysql.query("INSERT INTO pools (id, ip, last_checkin, active, hostname) VALUES (?, ?, now(), ?, ?) ON DUPLICATE KEY UPDATE last_checkin=now(), active=?", - [global.config.pool_id, global.config.bind_ip, true, global.config.hostname, true]); - global.mysql.query("DELETE FROM ports WHERE pool_id = ?", [global.config.pool_id]).then(function () { - global.config.ports.forEach(function (port) { - if ('ssl' in port && port.ssl === true) { - global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 1)", - [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); - } else { - global.mysql.query("INSERT INTO ports (pool_id, network_port, starting_diff, port_type, description, hidden, ip_address, ssl_port) values (?, ?, ?, ?, ?, ?, ?, 0)", - [global.config.pool_id, port.port, port.difficulty, port.portType, port.desc, port.hidden, global.config.bind_ip]); - } - }); - }); }); } @@ -79,27 +146,20 @@ function messageHandler(message) { if (cluster.isMaster) { sendToWorkers(message); } else { - bannedIPs.push(message.data); + if (!localhostCheck.test(message.data)) bannedTmpIPs[message.data] = 1; + else if (message.wallet) bannedTmpWallets[message.wallet] = 1; } break; case 'newBlockTemplate': debug(threadName + "Received new block template"); - if (cluster.isMaster) { - sendToWorkers(message); - newBlockTemplate(message.data); - } else { - newBlockTemplate(message.data); - } + setNewBlockTemplate(message.data); break; - case 'removeMiner': - if (cluster.isMaster) { - minerCount[message.data] -= 1; - } + case 'newCoinHashFactor': + debug(threadName + "Received new coin hash factor"); + setNewCoinHashFactor(true, message.data.coin, message.data.coinHashFactor); break; - case 'newMiner': - if (cluster.isMaster) { - minerCount[message.data] += 1; - } + case 'minerPortCount': + if (cluster.isMaster) minerCount[message.data.worker_id] = message.data.ports; break; case 'sendRemote': if (cluster.isMaster) { @@ -107,740 +167,2303 @@ function messageHandler(message) { } break; case 'trustedShare': - trustedShares += 1; - totalShares += 1; + ++ trustedShares; + ++ totalShares; break; case 'normalShare': - normalShares += 1; - totalShares += 1; + ++ normalShares; + ++ totalShares; break; case 'invalidShare': - invalidShares += 1; - totalShares += 1; + ++ invalidShares; + ++ totalShares; + break; + case 'outdatedShare': + ++ outdatedShares; + // total shares will be also increased separately as part of share type above + break; + case 'throttledShare': + ++ throttledShares; + ++ totalShares; + break; } } process.on('message', messageHandler); function sendToWorkers(data) { - workerList.forEach(function (worker) { - worker.send(data); + Object.keys(cluster.workers).forEach(function(key) { + cluster.workers[key].send(data); }); } +function adjustMinerDiff(miner) { + if (miner.fixed_diff) { + const newDiff = miner.calcNewDiff(); + if (miner.difficulty * 10 < newDiff) { + console.log("Dropped low fixed diff " + miner.difficulty + " for " + miner.logString + " miner to " + newDiff + " dynamic diff"); + miner.fixed_diff = false; + if (miner.setNewDiff(newDiff)) return true; + } + } else if (miner.setNewDiff(miner.calcNewDiff())) { + return true; + } + return false; +} + function retargetMiners() { debug(threadName + "Performing difficulty check on miners"); - console.log('Performing difficulty update on miners'); - for (let minerId in activeMiners) { - if (activeMiners.hasOwnProperty(minerId)) { - let miner = activeMiners[minerId]; - if (!miner.fixed_diff) { - miner.updateDifficulty(); - } + + global.config.ports.forEach(function (portData) { minerCount[portData.port] = 0; }); + const time_before = Date.now(); + for (var [minerId, miner] of activeMiners) { + if (adjustMinerDiff(miner)) miner.sendSameCoinJob(); + ++ minerCount[miner.port]; + } + const elapsed = Date.now() - time_before; + if (elapsed > 50) console.error(threadName + "retargetMiners() consumed " + elapsed + " ms for " + activeMiners.size + " miners"); + process.send({type: 'minerPortCount', data: { worker_id: process.env['WORKER_ID'], ports: minerCount } }); +} + +// wallet " " proxy miner name -> { connectTime, count (miner), hashes } +// this is needed to set cummulative based diff for workers provided by Atreides proxy and xmrig-proxy +let proxyMiners = {}; + +function addProxyMiner(miner) { + if (miner.proxyMinerName && miner.proxyMinerName in proxyMiners) return; + + const wallet = miner.payout; + const proxyMinerName = wallet; //+ ":" + miner.identifier; + miner.proxyMinerName = proxyMinerName; + + if (!(proxyMinerName in proxyMiners)) { + proxyMiners[proxyMinerName] = {}; + proxyMiners[proxyMinerName].connectTime = Date.now(); + proxyMiners[proxyMinerName].count = 1; + proxyMiners[proxyMinerName].hashes = 0; + console.log("Starting to calculate high diff for " + proxyMinerName + " proxy"); + } else { + if (++ proxyMiners[proxyMinerName].count > global.config.pool.workerMax && !miner.xmrig_proxy) { + console.error(threadName + "Starting to long ban " + wallet + " miner address"); + bannedBigTmpWallets[wallet] = 1; + for (var [minerId2, miner2] of activeMiners) if (miner2.payout === wallet) removeMiner(miner2); + return false; } } + return true; +} + +function removeMiner(miner) { + if (!miner || miner.removed_miner) return; + const proxyMinerName = miner.proxyMinerName; + if (proxyMinerName && proxyMinerName in proxyMiners && --proxyMiners[proxyMinerName].count <= 0) delete proxyMiners[proxyMinerName]; + if (miner.payout in minerWallets && --minerWallets[miner.payout].count <= 0) delete minerWallets[miner.payout]; + activeMiners.delete(miner.id); + miner.removed_miner = true; } function checkAliveMiners() { debug(threadName + "Verifying if miners are still alive"); - for (let minerId in activeMiners) { - if (activeMiners.hasOwnProperty(minerId)) { - let miner = activeMiners[minerId]; - if (Date.now() - miner.lastContact > global.config.pool.minerTimeout * 1000) { - process.send({type: 'removeMiner', data: miner.port}); - delete activeMiners[minerId]; - } + const time_before = Date.now(); + const deadline = time_before - global.config.pool.minerTimeout * 1000; + for (var [minerId, miner] of activeMiners) if (miner.lastContact < deadline) removeMiner(miner); + const elapsed = Date.now() - time_before; + if (elapsed > 50) console.error(threadName + "checkAliveMiners() consumed " + elapsed + " ms for " + activeMiners.size + " miners"); +} + +// coin hash factor is only updated in master thread +function updateCoinHashFactor(coin) { + global.support.getCoinHashFactor(coin, function (coinHashFactor) { + if (coinHashFactor === null) { + console.error("Error getting coinHashFactor for " + coin + " coin"); + coinHashFactorUpdate(coin, newCoinHashFactor[coin] = 0); + } else if (!coinHashFactor) { + coinHashFactorUpdate(coin, newCoinHashFactor[coin] = 0); + } else { + newCoinHashFactor[coin] = coinHashFactor; + } + }); +} + +function process_rpc_template(rpc_template, coin, port, coinHashFactor, isHashFactorChange) { + let template = Object.assign({}, rpc_template); + + template.coin = coin; + template.port = parseInt(port); + template.coinHashFactor = coinHashFactor; + template.isHashFactorChange = isHashFactorChange; + + if (port in global.coinFuncs.getMM_PORTS()) { + const child_coin = global.coinFuncs.PORT2COIN(global.coinFuncs.getMM_PORTS()[port]); + if (child_coin in activeBlockTemplates) { + template.child_template = activeBlockTemplates[child_coin]; + template.child_template_buffer = template.child_template.buffer; + template.parent_blocktemplate_blob = global.coinFuncs.constructMMParentBlockBlob( + Buffer.from(rpc_template.blocktemplate_blob, 'hex'), port, template.child_template_buffer + ).toString('hex'); } } + + return template; } -function templateUpdate(repeating) { - global.coinFuncs.getBlockTemplate(global.config.pool.address, function (rpcResponse) { - if (rpcResponse && typeof rpcResponse.result !== 'undefined') { - rpcResponse = rpcResponse.result; - let buffer = new Buffer(rpcResponse.blocktemplate_blob, 'hex'); - let new_hash = new Buffer(32); - buffer.copy(new_hash, 0, 7, 39); - if (!activeBlockTemplate || new_hash.toString('hex') !== activeBlockTemplate.previous_hash.toString('hex')) { - debug(threadName + "New block template found at " + rpcResponse.height + " height with hash: " + new_hash.toString('hex')); - if (cluster.isMaster) { - sendToWorkers({type: 'newBlockTemplate', data: rpcResponse}); - newBlockTemplate(rpcResponse); - } else { - process.send({type: 'newBlockTemplate', data: rpcResponse}); - newBlockTemplate(rpcResponse); +// templateUpdate3 is only called in master thread (except the beginning of a worker thread) +function templateUpdate3(coin, port, coinHashFactor, isHashFactorChange, body_bt) { + const template = process_rpc_template(body_bt, coin, port, coinHashFactor, isHashFactorChange); + debug(threadName + "New block template found at " + template.height + " height"); + if (cluster.isMaster) { + sendToWorkers({type: 'newBlockTemplate', data: template}); + setNewBlockTemplate(template); + // update parent coins if current coin was updated now + if (port in global.coinFuncs.getMM_CHILD_PORTS()) { + const parent_ports = global.coinFuncs.getMM_CHILD_PORTS()[port]; + for (let parent_port in parent_ports) { + const parent_coin = global.coinFuncs.PORT2COIN(parent_port); + if (parent_coin in activeBlockTemplates) { + const parent_template = process_rpc_template(activeBlockTemplates[parent_coin], parent_coin, parent_port, lastCoinHashFactor[parent_coin], false); + sendToWorkers({type: 'newBlockTemplate', data: parent_template}); + setNewBlockTemplate(parent_template); } } + } + } else { + setNewBlockTemplate(template); + } +} + +// templateUpdate2 is only called in master thread (except the beginning of a worker thread) +function templateUpdate2(coin, port, isHashChange, coinHashFactor, isHashFactorChange, body_header) { + if (port == 8545 || port == 8645) { + return templateUpdate3(coin, port, coinHashFactor, isHashFactorChange, body_header); + } else global.coinFuncs.getPortBlockTemplate(port, function (body_bt) { + if (!newCoinHashFactor[coin]) { + console.log("Aborting " + port + " last block template request because " + coin + " already has zero hash factor"); + return; + } + if (!body_bt) { + console.error("Block template request failed for " + port + " port"); + coinHashFactorUpdate(coin, 0); + return; + } + + const time_now = Date.now(); + const maxBlockKeepTime = ("maxBlockKeepTime" + coin in global.config.daemon ? global.config.daemon["maxBlockKeepTime" + coin] : 60*60) * 1000; + const isTimeChange = !(coin in lastBlockKeepTime) || time_now - lastBlockKeepTime[coin] > maxBlockKeepTime; + const isRewardCheckReady = body_bt.expected_reward && (coin in lastBlockReward) && lastBlockReward[coin]; + const isRewardChange = isRewardCheckReady && body_bt.expected_reward / lastBlockReward[coin] > 1.01; + + if (isHashChange || (isTimeChange && (!isRewardCheckReady || body_bt.expected_reward !== lastBlockReward[coin])) || isRewardChange) { + lastBlockKeepTime[coin] = time_now; + lastBlockReward[coin] = body_bt.expected_reward; + return templateUpdate3(coin, port, coinHashFactor, isHashFactorChange, body_bt); + } + }); +} + +function coinHashFactorUpdate(coin, coinHashFactor) { + if (coin === "") return; + if (coinHashFactor === 0 && lastCoinHashFactor[coin] === 0) return; + if (cluster.isMaster) { + //console.log('[*] New ' + coin + ' coin hash factor is set from ' + newCoinHashFactor[coin] + ' to ' + coinHashFactor); + let data = { coin: coin, coinHashFactor: coinHashFactor }; + sendToWorkers({type: 'newCoinHashFactor', data: data}); + } + setNewCoinHashFactor(true, coin, coinHashFactor); +} + +// templateUpdate is only called in master thread (except the beginning of a worker thread) +function templateUpdate(coin, repeating) { + const port = global.coinFuncs.COIN2PORT(coin); + const coinHashFactor = newCoinHashFactor[coin]; + if (coinHashFactor) { + global.coinFuncs.getPortLastBlockHeader(port, function (err, body) { + if (!newCoinHashFactor[coin]) { + console.log(threadName + "Aborting " + port + " last block header request because " + coin + " already has zero hash factor"); + if (repeating === true) setTimeout(templateUpdate, DAEMON_POLL_MS, coin, repeating); + } else if (err === null && body.hash) { + const isHashFactorChange = Math.abs(lastCoinHashFactor[coin] - coinHashFactor) / coinHashFactor > 0.05; + const pollBlockInterval = "pollBlockInterval" + coin in global.config.daemon ? global.config.daemon["pollBlockInterval" + coin] : 60*60*1000; + const time_now = Date.now(); + const isHashChange = !(coin in lastBlockHash) || body.hash !== lastBlockHash[coin]; + const isTimeChange = !(coin in lastBlockTime) || time_now - lastBlockTime[coin] > pollBlockInterval; + if ( isHashChange || isTimeChange) { + lastBlockHash[coin] = body.hash; + lastBlockTime[coin] = time_now; + templateUpdate2(coin, port, isHashChange, coinHashFactor, isHashFactorChange, body); + } else if (isHashFactorChange) { + coinHashFactorUpdate(coin, coinHashFactor); + } + if (repeating === true) setTimeout(templateUpdate, DAEMON_POLL_MS, coin, repeating); } else { - if (repeating !== true) { - setTimeout(templateUpdate, 300); + console.error(threadName + "Last block header request for " + port + " port failed!"); + coinHashFactorUpdate(coin, 0); + if (repeating !== false) setTimeout(templateUpdate, global.config.daemon.pollInterval, coin, repeating); + } + }); + } else if (cluster.isMaster) { + if (repeating !== false) setTimeout(templateUpdate, global.config.daemon.pollInterval, coin, repeating); + } +} + +// main chain anchor block height for alt chain block +let anchorBlockHeight; +let anchorBlockPrevHeight; + +// update main chain anchor block height for alt chain block +// anchorBlockUpdate is only called in worker threads +function anchorBlockUpdate() { + if (("" in activeBlockTemplates) && global.config.daemon.port == activeBlockTemplates[""].port) return; + // only need to do that separately if we mine alt chain + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err === null) { + anchorBlockHeight = body.height + 1; + if (!anchorBlockPrevHeight || anchorBlockPrevHeight != anchorBlockHeight) { + anchorBlockPrevHeight = anchorBlockHeight; + debug("Anchor block was changed to " + anchorBlockHeight); } + } else { + console.error("Archor last block header request failed!"); } }); } -function newBlockTemplate(template) { - let buffer = new Buffer(template.blocktemplate_blob, 'hex'); - let previous_hash = new Buffer(32); - buffer.copy(previous_hash, 0, 7, 39); - console.log(threadName + 'New block to mine at height: ' + template.height + '. Difficulty: ' + template.difficulty); - if (activeBlockTemplate) { - pastBlockTemplates.enq(activeBlockTemplate); +function getCoinJobParams(coin) { + let params = {}; + params.bt = activeBlockTemplates[coin]; + params.coinHashFactor = lastCoinHashFactorMM[coin]; + params.algo_name = global.coinFuncs.algoShortTypeStr(params.bt.port, params.bt.block_version); + //params.variant_name = params.algo_name.split('/')[1]; + return params; +}; + +function setNewCoinHashFactor(isHashFactorChange, coin, coinHashFactor, check_height) { + if (isHashFactorChange) lastCoinHashFactor[coin] = coinHashFactor; + const prevCoinHashFactorMM = lastCoinHashFactorMM[coin]; + lastCoinHashFactorMM[coin] = coinHashFactor; // used in miner.selectBestCoin + + const port = global.coinFuncs.COIN2PORT(coin); + const is_mm = port in global.coinFuncs.getMM_PORTS(); + if (is_mm) { + const child_coin = global.coinFuncs.PORT2COIN(global.coinFuncs.getMM_PORTS()[port]); + lastCoinHashFactorMM[coin] += lastCoinHashFactor[child_coin]; + } + + if (cluster.isMaster && coin !== "" && prevCoinHashFactorMM != lastCoinHashFactorMM[coin]) { + console.log('[*] New ' + coin + ' coin hash factor is set from ' + prevCoinHashFactorMM + ' to ' + coinHashFactor + (is_mm ? ' (MM: ' + lastCoinHashFactorMM[coin] + ')' : "")); } - activeBlockTemplate = new BlockTemplate(template); - for (let minerId in activeMiners) { - if (activeMiners.hasOwnProperty(minerId)) { - let miner = activeMiners[minerId]; - debug(threadName + "Updating worker " + miner.payout + " With new work at height: " + template.height); - miner.sendNewJob(); + if (!(coin in activeBlockTemplates)) return; + + // update parent coins if current coin was updated now + if (isHashFactorChange) if (port in global.coinFuncs.getMM_CHILD_PORTS()) { + const parent_ports = global.coinFuncs.getMM_CHILD_PORTS()[port]; + for (let parent_port in parent_ports) { + const parent_coin = global.coinFuncs.PORT2COIN(parent_port); + setNewCoinHashFactor(true, parent_coin, lastCoinHashFactor[parent_coin], 0); } } + + const time_before = Date.now(); + let strLogPrefix; + + if (isHashFactorChange) { + const port = activeBlockTemplates[coin].port; + const block_version = activeBlockTemplates[coin].block_version; + const algo = global.coinFuncs.algoShortTypeStr(port, block_version); + + strLogPrefix = "Full BT update for coin " + coin; + if (cluster.isMaster) console.log(threadName + strLogPrefix + " with hash factor changed to " + lastCoinHashFactorMM[coin]); + + if (check_height) { + for (var [minerId, miner] of activeMiners) { + if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) continue; + miner.trust.check_height = check_height; + miner.sendBestCoinJob(); + } + } else { + for (var [minerId, miner] of activeMiners) { + if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) continue; + miner.sendBestCoinJob(); + } + } + + } else { + + strLogPrefix = "Fast BT update for coin " + coin; + if (cluster.isMaster) console.log(threadName + strLogPrefix + " with the same " + lastCoinHashFactorMM[coin] + " hash factor"); + + const params = getCoinJobParams(coin); + if (check_height) { + for (var [minerId, miner] of activeMiners) { + //if (typeof(miner.curr_coin) === 'undefined') console.error("[INTERNAL ERROR]: " + miner.logString + ": undefined curr_coin"); + if (miner.curr_coin !== coin) continue; + //if (!(coin in miner.coin_perf)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported coin " + coin + " in miner " + JSON.stringify(miner.coin_perf) + " coin_perf"); + //if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported algo " + algo + " in miner " + JSON.stringify(miner.algos) + " algos"); + miner.trust.check_height = check_height; + miner.sendCoinJob(coin, params); + } + } else { + for (var [minerId, miner] of activeMiners) { + //if (typeof(miner.curr_coin) === 'undefined') console.error("[INTERNAL ERROR]: " + miner.logString + ": undefined curr_coin"); + if (miner.curr_coin !== coin) continue; + //if (!(coin in miner.coin_perf)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported coin " + coin + " in miner " + JSON.stringify(miner.coin_perf) + " coin_perf"); + //if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) console.error("[INTERNAL ERROR]: " + miner.logString + ": no longer supported algo " + algo + " in miner " + JSON.stringify(miner.algos) + " algos"); + miner.sendCoinJob(coin, params); + } + } + } + + const elapsed = Date.now() - time_before; + if (elapsed > 50) console.error(threadName + strLogPrefix + " setNewCoinHashFactor() consumed " + elapsed + " ms for " + activeMiners.size + " miners"); } -let VarDiff = (function () { - let variance = global.config.pool.varDiffVariance / 100 * global.config.pool.targetTime; - return { - tMin: global.config.pool.targetTime - variance, - tMax: global.config.pool.targetTime + variance - }; -})(); +function setNewBlockTemplate(template) { + const coin = template.coin; + let isExtraCheck = false; + if (coin in activeBlockTemplates) { + if (coin in pastBlockTemplates) { + pastBlockTemplates[coin].get(0).timeoutTime = Date.now() + 4*1000; + } else { + pastBlockTemplates[coin] = global.support.circularBuffer(10); + } + pastBlockTemplates[coin].enq(activeBlockTemplates[coin]); + if (activeBlockTemplates[coin].port != template.port && global.config.pool.trustedMiners) isExtraCheck = true; + } + if (cluster.isMaster) { + const coin_str = coin === "" ? "" : coin + " "; + console.log('[*] New ' + coin_str + 'block to mine at ' + template.height + ' height with ' + template.difficulty + ' difficulty and ' + template.port + ' port (with coin hash factor ' + template.coinHashFactor + ")"); + } else { + debug(threadName + 'New block to mine at ' + template.height + ' height with ' + template.difficulty + ' difficulty and ' + template.port + ' port'); + } -function Miner(id, login, pass, ipAddress, startingDiff, messageSender, protoVersion, portType, port, agent) { - // Username Layout -
. - // Password Layout - .. - // Default function is to use the password so they can login. Identifiers can be unique, payment ID is last. - // If there is no miner identifier, then the miner identifier is set to the password - // If the password is x, aka, old-logins, we're not going to allow detailed review of miners. + activeBlockTemplates[coin] = new global.coinFuncs.BlockTemplate(template); + activeBlockTemplates[coin].timeCreated = Date.now(); - // Miner Variables - let pass_split = pass.split(":"); - this.error = ""; - this.identifier = pass_split[0]; - this.proxy = false; - if (agent && agent.includes('MinerGate')) { - this.identifier = "MinerGate"; + const height = activeBlockTemplates[coin].height; + + if (coin === "" && global.config.daemon.port == activeBlockTemplates[""].port) { + anchorBlockHeight = height; } - if (agent && agent.includes('xmr-node-proxy')) { - this.proxy = true; + + setNewCoinHashFactor(template.isHashFactorChange, coin, template.coinHashFactor, isExtraCheck ? height : 0); +} + +// here we keep verified share number of a specific wallet (miner.payout) +// it will reset to 0 after invalid share is found +// if walletTrust exceeds certain threshold (global.config.pool.trustThreshold * 100) then low diff (<=16000) new workers for this wallet are started with high trust +// this is needed to avoid CPU overload after constant miner reconnections that happen during mining botnet swarms +let walletTrust = {}; +// wallet last seen time (all wallets that are not detected for more than 1 day are removed) +let walletLastSeeTime = {}; + +// miner agent strings (for process.env['WORKER_ID'] == 1) +let minerAgents = {}; + +var reEmail = /^\S+@\S+\.\S+$/; +// wallet password last check time +let walletLastCheckTime = {}; + +// Print extra info for these wallets +let wallet_debug = {}; +// Do not ban from these IPs +let ip_whitelist = {}; + +function getTargetHex(diff, size) { + return pad_hex(baseDiff.div(diff).toBuffer({endian: 'little', size: size}).toString('hex'), size); +}; + +function getRavenTargetHex(diff) { + return pad_hex((baseRavenDiff / diff).toString(16), 32); +}; + +function Miner(id, login, pass, rigid, ipAddress, startingDiff, pushMessage, protoVersion, portType, port, agent, algos, algos_perf, algo_min_time) { + // Username Layout: monero_address[.payment_id][(%N%monero_address_95char)+][+difficulty_number] + // Password Layout: worker_name[:email_or_pass[:monero_address]][~algo_name] + // If email_or_pass is email then miners will get email notifications about payments and offline workers + + const login_diff_split = login.split("+"); + const login_div_split = login_diff_split[0].split("%"); + const login_paymentid_split = login_div_split[0].split("."); + const pass_algo_split = pass.split("~"); + let pass_split = pass_algo_split[0].split(":"); + + // Workaround for a common mistake to put email without : before it + // and also security measure to hide emails used as worker names + if (pass_split.length === 1 && reEmail.test(pass_split[0])) { + pass_split.push(pass_split[0]); + pass_split[0] = "email"; } + + // 1) set payout, identifier, email and logString + + this.payout = this.address = (pass_split.length === 3 ? pass_split[2] : login_paymentid_split[0]); this.paymentID = null; - this.valid_miner = true; - this.port = port; - this.portType = portType; - this.incremented = false; - switch (portType) { - case 'pplns': - this.poolTypeEnum = global.protos.POOLTYPE.PPLNS; - break; - case 'pps': - this.poolTypeEnum = global.protos.POOLTYPE.PPS; - break; - case 'solo': - this.poolTypeEnum = global.protos.POOLTYPE.SOLO; - break; - case 'prop': - this.poolTypeEnum = global.protos.POOLTYPE.PROP; - break; + + this.identifier = agent && agent.includes('MinerGate') ? "MinerGate" : (rigid ? rigid : pass_split[0]).substring(0, 64); + if (typeof(login_paymentid_split[1]) !== 'undefined') { + if (login_paymentid_split[1].length === 64 && hexMatch.test(login_paymentid_split[1]) && global.coinFuncs.validatePlainAddress(this.address)) { + this.paymentID = login_paymentid_split[1]; + this.payout += "." + this.paymentID; + if (typeof(login_paymentid_split[2]) !== 'undefined' && this.identifier === 'x') { + this.identifier = login_paymentid_split[2].substring(0, 64); + } + } else if (this.identifier === 'x') { + this.identifier = login_paymentid_split[1].substring(0, 64); + } } - let diffSplit = login.split("+"); - let addressSplit = diffSplit[0].split('.'); - this.address = addressSplit[0]; - this.payout = addressSplit[0]; - this.fixed_diff = false; - this.difficulty = startingDiff; - this.connectTime = Date.now(); - if (agent && agent.includes('NiceHash')) { - this.fixed_diff = true; - this.difficulty = global.coinFuncs.niceHashDiff; + + this.debugMiner = this.payout in wallet_debug; + this.whiteList = ipAddress in ip_whitelist; + + this.email = pass_split.length >= 2 ? pass_split[1] : ""; + this.logString = this.payout.substr(this.payout.length - 10) + ":" + this.identifier + " (" + ipAddress + ")"; + this.agent = agent; + + // 2) check stuff + + if (login_diff_split.length > 2) { + this.error = "Please use monero_address[.payment_id][(%N%monero_address_95char)+][+difficulty_number] login/user format"; + this.valid_miner = false; + return; } - if (diffSplit.length === 2) { - this.fixed_diff = true; - this.difficulty = Number(diffSplit[1]); - if (this.difficulty < global.config.pool.minDifficulty) { - this.difficulty = global.config.pool.minDifficulty; - } - if (this.difficulty > global.config.pool.maxDifficulty) { - this.difficulty = global.config.pool.maxDifficulty; - } - } else if (diffSplit.length > 2) { - this.error = "Too many options in the login field"; + + if (Math.abs(login_div_split.length % 2) == 0 || login_div_split.length > 5) { + this.error = "Please use monero_address[.payment_id][(%N%monero_address_95char)+][+difficulty_number] login/user format"; this.valid_miner = false; + return; } - if (typeof(addressSplit[1]) !== 'undefined' && addressSplit[1].length === 64 && hexMatch.test(addressSplit[1])) { - this.paymentID = addressSplit[1]; - this.payout = this.address + "." + this.paymentID; - } else if (typeof(addressSplit[1]) !== 'undefined') { - this.identifier = pass_split[0] === 'x' ? addressSplit[1] : pass_split[0]; + + this.payout_div = {}; + + let payout_percent_left = 100; + for (let index = 1; index < login_div_split.length - 1; index += 2) { + const percent = parseFloat(login_div_split[index]); + if (isNaN(percent) || percent < 0.1) { + this.error = "Your payment divide split " + percent + " is below 0.1% and can't be processed"; + this.valid_miner = false; + return; + } + if (percent > 99.9) { + this.error = "Your payment divide split " + percent + " is above 99.9% and can't be processed"; + this.valid_miner = false; + return; + } + payout_percent_left -= percent; + if (payout_percent_left < 0.1) { + this.error = "Your summary payment divide split exceeds 99.9% and can't be processed"; + this.valid_miner = false; + return; + } + const address = login_div_split[index + 1]; + if (address.length != 95 || !global.coinFuncs.validateAddress(address)) { + this.error = "Invalid payment address provided: " + address + ". Please use 95_char_long_monero_wallet_address format"; + this.valid_miner = false; + return; + } + if (address in bannedAddresses) { // Banned Address + this.error = "Permanently banned payment address " + address + " provided: " + bannedAddresses[address]; + this.valid_miner = false; + return; + } + if (address in bannedTmpWallets) { + this.error = "Temporary (10 minutes max) banned payment address " + address; + this.valid_miner = false; + return; + } + if (address in bannedBigTmpWallets) { + this.error = "Temporary (one hour max) ban since you connected too many workers. Please use proxy (https://github.com/MoneroOcean/xmrig-proxy)"; + this.valid_miner = false; + this.delay_reply = 600; + return; + } + if (address in this.payout_div) { + this.error = "You can't repeat payment split address " + address; + this.valid_miner = false; + return; + } + this.payout_div[address] = percent; } - if (typeof(addressSplit[2]) !== 'undefined') { - this.identifier = pass_split[0] === 'x' ? addressSplit[2] : pass_split[0]; + + if (payout_percent_left === 100) { + this.payout_div = null; + } else { + if (this.payout in this.payout_div) { + this.error = "You can't repeat payment split address " + this.payout; + this.valid_miner = false; + return; + } + this.payout_div[this.payout] = payout_percent_left; } - if (pass_split.length === 2) { - /* - Email address is: pass_split[1] - Need to do an initial registration call here. Might as well do it right... - */ - let payoutAddress = this.payout; - global.mysql.query("SELECT id FROM users WHERE username = ? LIMIT 1", [this.payout]).then(function (rows) { - if (rows.length > 0) { - return; - } - if (global.coinFuncs.blockedAddresses.indexOf(payoutAddress) !== -1) { - return; - } - global.mysql.query("INSERT INTO users (username, email) VALUES (?, ?)", [payoutAddress, pass_split[1]]); - }); - } else if (pass_split.length > 2) { - this.error = "Too many options in the password field"; + if (pass_split.length > 3) { + this.error = "Please use worker_name[:email_or_pass[:monero_address]][~algo_name] password format"; this.valid_miner = false; + return; } - if (global.coinFuncs.validateAddress(this.address)) { - this.bitcoin = 0; - } else if (btcValidator.validate(this.address) && global.config.general.allowBitcoin && global.coinFuncs.supportsAutoExchange) { - this.bitcoin = 1; - } else if (btcValidator.validate(this.address)) { - this.error = "This pool does not allow payouts to bitcoin."; + if (this.payout in bannedAddresses) { // Banned Address + this.error = "Permanently banned payment address " + this.payout + " provided: " + bannedAddresses[this.payout]; this.valid_miner = false; - } else { - // Invalid Addresses - this.error = "Invalid payment address provided"; + return; + } + + if (this.payout in bannedTmpWallets) { + this.error = "Temporary (10 minutes max) banned payment address " + this.payout; this.valid_miner = false; + return; } - if (bannedAddresses.indexOf(this.address) !== -1) { - // Banned Address - this.error = "Banned payment address provided"; + + if (this.payout in bannedBigTmpWallets) { + this.error = "Temporary (one hour max) ban since you connected too many workers. Please use proxy (https://github.com/MoneroOcean/xmrig-proxy)"; this.valid_miner = false; + this.delay_reply = 600; + return; } + if (global.coinFuncs.exchangeAddresses.indexOf(this.address) !== -1 && !(this.paymentID)) { - this.error = "Exchange addresses need payment IDs"; + this.error = "Exchange addresses need 64 hex character long payment IDs. Please specify it after your wallet address as follows after dot: Wallet.PaymentID"; + this.valid_miner = false; + return; + } + + if (!global.coinFuncs.validateAddress(this.address)) { + this.error = "Invalid payment address provided: " + this.address + ". Please use 95_char_long_monero_wallet_address format"; this.valid_miner = false; + return; } - if (!activeBlockTemplate) { + + if (!("" in activeBlockTemplates)) { this.error = "No active block template"; this.valid_miner = false; + return; + } + + this.setAlgos = function(algos, algos_perf, algo_min_time) { + this.algos = {}; + for (let i in algos) this.algos[algos[i]] = 1; + if (global.coinFuncs.is_miner_agent_no_haven_support(this.agent)) delete this.algos["cn-heavy/xhv"]; + if (this.algos["kawpow4"]) { + this.algos["kawpow"] = 1; + delete this.algos["kawpow4"]; + } + const check = global.coinFuncs.algoCheck(this.algos); + if (check !== true) return check; + if ("cn-pico" in this.algos) this.algos["cn-pico/trtl"] = 1; + + if (!(algos_perf && algos_perf instanceof Object)) { + if (global.coinFuncs.algoMainCheck(this.algos)) algos_perf = global.coinFuncs.getDefaultAlgosPerf(); + else algos_perf = global.coinFuncs.getPrevAlgosPerf(); + } + + let coin_perf = global.coinFuncs.convertAlgosToCoinPerf(algos_perf); + if (coin_perf instanceof Object) { + if (!("" in coin_perf && global.coinFuncs.algoMainCheck(this.algos))) coin_perf[""] = -1; + this.coin_perf = coin_perf; + } else { + return coin_perf; + } + this.algo_min_time = algo_min_time ? algo_min_time : 60; + return ""; + }; + + if (pass_algo_split.length == 2) { + const algo_name = pass_algo_split[1]; + algos = [ algo_name ]; + algos_perf = {}; + algos_perf[algo_name] = 1; + algo_min_time = 60; + + } else if (!(algos && algos instanceof Array && global.config.daemon.enableAlgoSwitching)) { + const agent_algo = global.coinFuncs.get_miner_agent_not_supported_algo(agent); + if (agent_algo) { + algos = [ agent_algo ]; + } else { + algos = global.coinFuncs.getDefaultAlgos(); + } + algos_perf = global.coinFuncs.getDefaultAlgosPerf(); + algo_min_time = 60; + } + + const status = this.setAlgos(algos, algos_perf, algo_min_time); + if (status != "") { + this.error = status; + this.valid_miner = false; + return; } + + // 3) setup valid miner stuff + + // 3a) misc stuff + + this.error = ""; + this.valid_miner = true; + this.removed_miner = false; + + this.proxy = agent && agent.includes('xmr-node-proxy'); + this.xmrig_proxy = agent && agent.includes('xmrig-proxy'); this.id = id; this.ipAddress = ipAddress; - this.messageSender = messageSender; - this.heartbeat = function () { - this.lastContact = Date.now(); - }; + this.pushMessage = pushMessage; + this.connectTime = Date.now(); + this.heartbeat = function () { this.lastContact = Date.now(); }; this.heartbeat(); - // VarDiff System - this.shareTimeBuffer = global.support.circularBuffer(8); - this.shareTimeBuffer.enq(global.config.pool.targetTime); - this.lastShareTime = Date.now() / 1000 || 0; + // 3b) port stuff + + this.port = port; + this.portType = portType; + switch (portType) { + case 'pplns': this.poolTypeEnum = global.protos.POOLTYPE.PPLNS; break; + case 'pps': this.poolTypeEnum = global.protos.POOLTYPE.PPS; break; + case 'solo': this.poolTypeEnum = global.protos.POOLTYPE.SOLO; break; + case 'prop': this.poolTypeEnum = global.protos.POOLTYPE.PROP; break; + default: console.error("Wrong portType " + portType); + this.poolTypeEnum = global.protos.POOLTYPE.PPLNS; + } + + this.wallet_key = this.payout + " " + this.poolTypeEnum + " " + JSON.stringify(this.payout_div) + " "; + // 3c) diff calc stuff + + this.lastShareTime = Math.floor(Date.now() / 1000); this.validShares = 0; this.invalidShares = 0; this.hashes = 0; - this.logString = this.address + " ID: " + this.identifier + " IP: " + this.ipAddress; + + // 3d) trust stuff if (global.config.pool.trustedMiners) { + if (!(this.payout in walletTrust)) { + walletTrust[this.payout] = 0; + walletLastSeeTime[this.payout] = Date.now(); + } this.trust = { - threshold: global.config.pool.trustThreshold, - probability: 256, - penalty: 0 + trust: 0, + check_height: 0 }; } - this.validJobs = global.support.circularBuffer(4); - this.sentJobs = global.support.circularBuffer(8); + // 3e) password setup stuff + let email = this.email.trim(); + if (email != "") { + // Need to do an initial registration call here. Might as well do it right... + let payoutAddress = this.payout; + let time_now = Date.now(); + if (!(payoutAddress in walletLastCheckTime) || time_now - walletLastCheckTime[payoutAddress] > 60*1000) { + global.mysql.query("SELECT id FROM users WHERE username = ? LIMIT 1", [payoutAddress]).then(function (rows) { + if (rows.length > 0) return; + if (global.coinFuncs.blockedAddresses.indexOf(payoutAddress) !== -1) return; + global.mysql.query("INSERT INTO users (username, email) VALUES (?, ?)", [payoutAddress, email]).catch(function (error) { + console.error("SQL query failed: " + error); + }); + console.log("Setting password " + email + " for " + payoutAddress); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); + walletLastCheckTime[payoutAddress] = time_now; + } + } + + this.validJobs = global.support.circularBuffer(10); this.cachedJob = null; - this.invalidShareProto = global.protos.InvalidShare.encode({ - paymentAddress: this.address, - paymentID: this.paymentID, - identifier: this.identifier - }); + this.storeInvalidShare = function() { + const time_now = Date.now(); + if (this.invalidShareCount) ++ this.invalidShareCount; + else this.invalidShareCount = 1; + if (!this.lastInvalidShareTime || time_now - this.lastInvalidShareTime > 10*60*1000) { + let _this = this; + global.database.storeInvalidShare(global.protos.InvalidShare.encode({ + paymentAddress: _this.address, + paymentID: _this.paymentID, + identifier: _this.identifier, + count: _this.invalidShareCount + })); + this.lastInvalidShareTime = time_now; + this.invalidShareCount = 0; + } + }; - // Support functions for how miners activate and run. - this.updateDifficultyOld = function () { - let now = Math.round(Date.now() / 1000); - let avg = this.shareTimeBuffer.average(this.lastShareTime); + this.setNewDiff = function (difficulty) { + if (this.fixed_diff) return false; + const newDiff = difficulty; + this.newDiffRecommendation = newDiff; + const ratio = Math.abs(newDiff - this.difficulty) / this.difficulty; + if (ratio < 0.2) return false; + this.newDiffToSet = newDiff; + + debug(threadName + "Difficulty change to: " + this.newDiffToSet + " For: " + this.logString); + if (this.hashes > 0) { + debug(threadName + "Hashes: " + this.hashes + " in: " + Math.floor((Date.now() - this.connectTime) / 1000) + " seconds gives: " + + Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) + " hashes/second or: " + + Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) * global.config.pool.targetTime + " difficulty versus: " + this.newDiffToSet); + } + return true; + }; - let sinceLast = now - this.lastShareTime; - let decreaser = sinceLast > VarDiff.tMax; + this.selectBestCoin = function() { + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] current coin is " + this.curr_coin); + if (typeof(this.curr_coin) !== 'undefined' && this.curr_coin_time && lastCoinHashFactorMM[this.curr_coin] && + Date.now() - this.curr_coin_time < this.algo_min_time*1000 + ) { + return this.curr_coin; + } + let best_coin = ""; + let best_coin_perf = this.coin_perf[""] * 1.1; + let miner = this; + COINS.forEach(function(coin) { + if (!(coin in miner.coin_perf)) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no coin_perf"); + return; + } + if (!(coin in activeBlockTemplates)) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no activeBlockTemplates"); + return; + } + const coinHashFactor = lastCoinHashFactorMM[coin]; + if (!coinHashFactor) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no coinHashFactor"); + return; + } + const bt = activeBlockTemplates[coin]; + const port = bt.port; + const block_version = bt.block_version; + const algo = global.coinFuncs.algoShortTypeStr(port, block_version); - let newDiff; - let direction; + if (miner.difficulty / coinHashFactor > bt.difficulty * 3) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] Rejected best " + coin + " coin due to high diff " + miner.difficulty + " " + coinHashFactor + " " + bt.difficulty); + return; + } + if (!global.coinFuncs.isMinerSupportAlgo(algo, miner.algos)) { + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": no algo support"); + return; + } + let coin_perf = miner.coin_perf[coin] * coinHashFactor; + if (miner.curr_coin === coin) coin_perf *= 1.05; + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] " + coin + ": " + coin_perf); + if (coin_perf > best_coin_perf) { + best_coin = coin; + best_coin_perf = coin_perf; + } + }); + if (best_coin_perf < 0) return false; + if (typeof(this.curr_coin) === 'undefined' || this.curr_coin != best_coin) { + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(best_coin)); + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + this.curr_coin_min_diff = 1; + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + this.curr_coin_min_diff = 0.01; + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + this.curr_coin_min_diff = 0.01 * 0x100000000; + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + this.curr_coin_min_diff = 0.01 * 0x100000000; + } else { + this.curr_coin_min_diff = global.config.pool.minDifficulty; + } + this.curr_coin = best_coin; + this.curr_coin_hash_factor = lastCoinHashFactorMM[best_coin]; + this.curr_coin_time = Date.now(); + if (global.config.pool.trustedMiners) this.trust.check_height = activeBlockTemplates[best_coin].height; + } + return best_coin; + }; + + // 3e) set diff stuff + + this.fixed_diff = false; + this.difficulty = startingDiff; - if (avg > VarDiff.tMax && this.difficulty > global.config.pool.minDifficulty) { - newDiff = global.config.pool.targetTime / avg * this.difficulty; - direction = -1; + if (login_diff_split.length === 2) { + this.fixed_diff = true; + if (login_diff_split[1].substring(0, 4) === 'perf') { + let perfDiff = 0; + if (this.coin_perf[""] > 2) { + perfDiff = Math.floor(this.coin_perf[""] * (global.config.pool.targetTime || 30)); + } + if (login_diff_split[1].substring(4, 8) === 'auto' || perfDiff === 0) { + this.fixed_diff = false; + } + this.difficulty = perfDiff || startingDiff; + } else { + this.difficulty = Number(login_diff_split[1]); } - else if (avg < VarDiff.tMin && this.difficulty < global.config.pool.maxDifficulty) { - newDiff = global.config.pool.targetTime / avg * this.difficulty; - direction = 1; + if (this.difficulty < global.config.pool.minDifficulty) { + this.difficulty = global.config.pool.minDifficulty; } - else { - return; + if (this.difficulty > global.config.pool.maxDifficulty) { + this.difficulty = global.config.pool.maxDifficulty; } + } + + this.curr_coin_hash_factor = 1; + this.curr_coin_min_diff = global.config.pool.minDifficulty; + this.curr_coin = this.selectBestCoin(); - if (Math.abs(newDiff - this.difficulty) / this.difficulty * 100 > global.config.pool.maxDiffChange) { - let change = global.config.pool.maxDiffChange / 100 * this.difficulty * direction; - newDiff = this.difficulty + change; + if (agent && agent.includes('NiceHash')) { + this.fixed_diff = true; + let minNiceHashDiff; + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(this.curr_coin)); + if ( global.coinFuncs.blobTypeRvn(blob_type_num) || + global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num) + ) { + minNiceHashDiff = global.coinFuncs.niceHashDiff * 50; + } else { + minNiceHashDiff = global.coinFuncs.niceHashDiff; + } + + if (this.difficulty < minNiceHashDiff) this.difficulty = minNiceHashDiff; + } + + this.calcNewDiff = function () { + let miner; + let target; + let min_diff; + let history_time; + const time_now = Date.now(); + const proxyMinerName = this.payout; // + ":" + this.identifier; + let proxyMiner = proxyMiners[proxyMinerName]; + if (proxyMiner && proxyMiner.hashes / (time_now - proxyMiner.connectTime) > this.difficulty) { + miner = proxyMiner; + target = 15; + min_diff = 10 * global.config.pool.minDifficulty; + history_time = 5; + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] calc proxy miner diff: " + miner.hashes + " / " + ((time_now - miner.connectTime) / 1000)); + } else if (this.payout in minerWallets && minerWallets[this.payout].last_ver_shares >= global.config.pool.minerThrottleSharePerSec * global.config.pool.minerThrottleShareWindow) { + miner = minerWallets[this.payout]; + target = 15; + min_diff = 10 * global.config.pool.minDifficulty; + history_time = 5; + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] calc throttled miner diff: " + miner.hashes + " / " + ((time_now - miner.connectTime) / 1000)); + } else { + miner = this; + target = this.proxy ? 15 : global.config.pool.targetTime; + min_diff = this.proxy ? 10 * global.config.pool.minDifficulty : global.config.pool.minDifficulty; + history_time = 60; + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] calc miner diff: " + miner.hashes + " / " + ((time_now - miner.connectTime) / 1000)); } + if (miner.connectTimeShift) { + const timeSinceLastShift = time_now - miner.connectTimeShift; + const timeWindow = history_time * 60 * 1000; + if (timeSinceLastShift > timeWindow) { + if (timeSinceLastShift > 2 * timeWindow) { // forget all + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] forget diff"); + miner.hashes = 0; + } else { + if (this.debugMiner) console.log(threadName + this.logString + " [WALLET DEBUG] diff window shift from " + miner.connectTimeShift + " and " + miner.hashesShift + " hashes"); + miner.hashes -= miner.hashesShift; + } + miner.connectTime = miner.connectTimeShift; + miner.connectTimeShift = time_now; + miner.hashesShift = miner.hashes; + } + } else { + miner.connectTimeShift = time_now; + miner.hashesShift = miner.hashes; + } + + let hashes = miner.hashes; + let period = (time_now - miner.connectTime) / 1000; - this.setNewDiff(newDiff); - this.shareTimeBuffer.clear(); - if (decreaser) { - this.lastShareTime = now; + if (hashes === 0) { + hashes = this.difficulty; + target = 2 * global.config.pool.retargetTime; + if (period < target) period = target; } + const diff = hashes * target / period; + return diff < min_diff ? min_diff : diff; }; - this.updateDifficulty = function () { - if (this.hashes > 0) { - let newDiff = 0; - if (this.proxy) { - newDiff = Math.floor(Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000)))* 5); + this.checkBan = function (validShare) { + if (!global.config.pool.banEnabled) return; + if (this.whiteList) return; + + // Valid stats are stored by the pool. + if (validShare) { + ++ this.validShares; + } else { + ++ this.invalidShares; + if (this.validShares === 0) { + console.error(threadName + "Suspended miner IP for submitting bad share with zero trust " + this.logString); + removeMiner(this); + process.send({type: 'banIP', data: this.ipAddress, wallet: this.payout}); + return; + } + } + + const shareCount = this.validShares + this.invalidShares; + if (shareCount >= global.config.pool.banThreshold) { + if (100 * this.invalidShares / shareCount >= global.config.pool.banPercent) { + console.error(threadName + "Suspended miner IP for submitting too many bad shares recently " + this.logString); + removeMiner(this); + process.send({type: 'banIP', data: this.ipAddress, wallet: this.payout}); } else { - newDiff = Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) * global.config.pool.targetTime; + this.invalidShares = 0; + this.validShares = 0; } - this.setNewDiff(newDiff); - } else { - this.updateDifficultyOld(); } }; - this.setNewDiff = function (difficulty) { - this.newDiff = Math.round(difficulty); - debug(threadName + "Difficulty: " + this.newDiff + " For: " + this.logString + " Time Average: " + this.shareTimeBuffer.average(this.lastShareTime) + " Entries: " + this.shareTimeBuffer.size() + " Sum: " + this.shareTimeBuffer.sum()); - if (this.newDiff > global.config.pool.maxDifficulty && !this.proxy) { - this.newDiff = global.config.pool.maxDifficulty; + if (protoVersion === 1) { + this.getCoinJob = function (coin, params) { + const bt = params.bt; + if (this.jobLastBlockHash === bt.idHash && !this.newDiffToSet && this.cachedJob !== null) return null; + this.jobLastBlockHash = bt.idHash; + + if (this.newDiffToSet) { + this.difficulty = this.newDiffToSet; + this.newDiffToSet = null; + this.newDiffRecommendation = null; + } else if (this.newDiffRecommendation) { + this.difficulty = this.newDiffRecommendation; + this.newDiffRecommendation = null; + } + + let coin_diff = this.difficulty / this.curr_coin_hash_factor; + if (coin_diff < this.curr_coin_min_diff) coin_diff = this.curr_coin_min_diff; + if (coin_diff > bt.difficulty) coin_diff = bt.difficulty; + + const blob_type_num = global.coinFuncs.portBlobType(bt.port); + const isEth = global.coinFuncs.blobTypeEth(blob_type_num); + const isErg = global.coinFuncs.blobTypeErg(blob_type_num); + const isExtraNonceBT = isEth || isErg; + + if (!this.proxy || isExtraNonceBT) { + const blob_hex = bt.nextBlobHex(); + if (!blob_hex) return null; + const isGrin = global.coinFuncs.blobTypeGrin(blob_type_num); + const isRvn = global.coinFuncs.blobTypeRvn(blob_type_num); + const newJob = { + id: isRvn ? get_new_eth_job_id() : get_new_id(), + coin: coin, + blob_type_num: blob_type_num, + blockHash: bt.idHash, + extraNonce: isExtraNonceBT ? this.eth_extranonce : bt.extraNonce, + height: bt.height, + seed_hash: bt.seed_hash, + difficulty: coin_diff, + norm_diff: coin_diff * this.curr_coin_hash_factor, + coinHashFactor: params.coinHashFactor, + submissions: {} + }; + this.validJobs.enq(newJob); + if (isGrin) this.cachedJob = { + pre_pow: blob_hex, + algo: this.protocol === "grin" ? "cuckaroo" : params.algo_name, + edgebits: 29, + proofsize: global.coinFuncs.c29ProofSize(blob_type_num), + noncebytes: 4, + height: bt.height, + job_id: newJob.id, + difficulty: coin_diff, + id: this.id + }; else if (isRvn) this.cachedJob = [ + newJob.id, + blob_hex, + bt.seed_hash, + getRavenTargetHex(coin_diff), + true, + bt.height, + bt.bits + ]; else if (isEth) this.cachedJob = [ + newJob.id, + bt.seed_hash, + blob_hex, + true, + coin_diff // this will be popped and used for separate mining.set_difficulty message + ]; else if (isErg) this.cachedJob = [ + newJob.id, + bt.height, + bt.hash, + "", + "", + 2, // curl http://localhost:9053/info: parameters.blockVersion + baseDiff.div(coin_diff).toString(), + "", + true + ]; else this.cachedJob = { + blob: blob_hex, + algo: params.algo_name, + height: bt.height, + seed_hash: bt.seed_hash, + job_id: newJob.id, + target: getTargetHex(coin_diff, global.coinFuncs.nonceSize(blob_type_num)), + id: this.id + }; + } else { + const blob_hex = bt.nextBlobWithChildNonceHex(); + const newJob = { + id: get_new_id(), + coin: coin, + blob_type_num: blob_type_num, + blockHash: bt.idHash, + extraNonce: bt.extraNonce, + height: bt.height, + seed_hash: bt.seed_hash, + difficulty: coin_diff, + norm_diff: coin_diff * this.curr_coin_hash_factor, + clientPoolLocation: bt.clientPoolLocation, + clientNonceLocation: bt.clientNonceLocation, + coinHashFactor: params.coinHashFactor, + submissions: {} + }; + this.validJobs.enq(newJob); + this.cachedJob = { + blocktemplate_blob: blob_hex, + blob_type: global.coinFuncs.blobTypeStr(bt.port, bt.block_version), + algo: params.algo_name, + difficulty: bt.difficulty, + height: bt.height, + seed_hash: bt.seed_hash, + reserved_offset: bt.reserved_offset, + client_nonce_offset: bt.clientNonceLocation, + client_pool_offset: bt.clientPoolLocation, + target_diff: coin_diff, + job_id: newJob.id, + id: this.id + }; + } + return this.cachedJob; + }; + + this.sendCoinJob = function(coin, params) { + const job = this.getCoinJob(coin, params); + if (job === null) return; + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(coin)); + if (this.protocol == "grin") { + return this.pushMessage({method: "getjobtemplate", result: job}); + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + const target = job[3]; + if (!this.last_target || this.last_target !== target) { + this.pushMessage({method: "mining.set_target", params: [ target ], id:null}); + this.last_target = target; + } + return this.pushMessage({method: "mining.notify", params: job, algo: params.algo_name, id:null}); + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + const diff = job.pop() / 0x100000000; + if (!this.last_diff || this.last_diff !== diff) { + this.pushMessage({method: "mining.set_difficulty", params: [ diff ]}); + this.last_diff = diff; + } + return this.pushMessage({method: "mining.notify", params: job, algo: params.algo_name}); + + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + return this.pushMessage({method: "mining.notify", params: job, algo: params.algo_name}); + + } else { + return this.pushMessage({method: "job", params: job}); + } + }; + + this.sendSameCoinJob = function () { + const coin = typeof(this.curr_coin) !== 'undefined' ? this.curr_coin : this.selectBestCoin(); + if (coin !== false) return this.sendCoinJob(coin, getCoinJobParams(coin)); + }; + + this.getBestCoinJob = function() { + const coin = this.selectBestCoin(); + if (coin !== false) return this.getCoinJob(coin, getCoinJobParams(coin)); + }; + + this.sendBestCoinJob = function() { + const coin = this.selectBestCoin(); + if (coin !== false) return this.sendCoinJob(coin, getCoinJobParams(coin)); + }; + } +} + +// store wallet_key (address, paymentID, poolTypeEnum, port) -> worker_name -> isTrustedShare -> (height, difficulty, time, acc, acc2) +let walletAcc = {}; +// number of worker_name for wallet_key (so we do not count them by iteration) +let walletWorkerCount = {}; +// is share finalizer function for dead worker_name is active +let is_walletAccFinalizer = {}; + +function storeShareDiv(miner, share_reward, share_reward2, share_num, worker_name, bt_port, bt_height, bt_difficulty, isBlockCandidate, isTrustedShare) { + const time_now = Date.now(); + if (miner.payout_div === null) { + global.database.storeShare(bt_height, global.protos.Share.encode({ + paymentAddress: miner.address, + paymentID: miner.paymentID, + raw_shares: share_reward, + shares2: share_reward2, + share_num: share_num, + identifier: worker_name, + port: bt_port, + blockHeight: bt_height, + blockDiff: bt_difficulty, + poolType: miner.poolTypeEnum, + foundBlock: isBlockCandidate, + trustedShare: isTrustedShare, + poolID: global.config.pool_id, + timestamp: time_now + })); + } else { + for (let payout in miner.payout_div) { + const payout_split = payout.split("."); + const paymentAddress = payout_split[0]; + const paymentID = payout_split.length === 2 ? payout_split[1] : null; + const payoutPercent = miner.payout_div[payout]; + const shares = share_reward * payoutPercent / 100; + const shares2 = Math.floor(share_reward2 * payoutPercent / 100); + global.database.storeShare(bt_height, global.protos.Share.encode({ + paymentAddress: paymentAddress, + paymentID: paymentID, + raw_shares: shares, + shares2: shares2, + share_num: share_num, + identifier: worker_name, + port: bt_port, + blockHeight: bt_height, + blockDiff: bt_difficulty, + poolType: miner.poolTypeEnum, + foundBlock: isBlockCandidate, + trustedShare: isTrustedShare, + poolID: global.config.pool_id, + timestamp: time_now + })); } - if (this.difficulty === this.newDiff) { - return; + } +} + +function walletAccFinalizer(wallet_key, miner, bt_port) { + debug("!!! " + wallet_key + ": scanning for old worker names"); + let wallet = walletAcc[wallet_key]; + let is_something_left = false; + let time_now = Date.now(); + for (let worker_name in wallet) { + let worker = wallet[worker_name]; + if (time_now - worker.time > global.config.pool.shareAccTime*1000) { + let acc = worker.acc; + if (acc != 0) { + let height = worker.height; + debug("!!! " + wallet_key + " / " + worker_name + ": storing old worker share " + height + " " + worker.difficulty + " " + time_now + " " + acc); + storeShareDiv(miner, acc, worker.acc2, worker.share_num, worker_name, bt_port, height, worker.difficulty, false, true); + } + debug("!!! " + wallet_key + ": removing old worker " + worker_name); + if (worker_name !== "all_other_workers") -- walletWorkerCount[wallet_key]; + delete wallet[worker_name]; + } else { + is_something_left = true; + } + } + + if (is_something_left) { + setTimeout(walletAccFinalizer, global.config.pool.shareAccTime*1000, wallet_key, miner, bt_port); + } else { + is_walletAccFinalizer[wallet_key] = false; + } +} + +function recordShareData(miner, job, isTrustedShare, blockTemplate) { + miner.hashes += job.norm_diff; + let proxyMinerName = miner.payout; // + ":" + miner.identifier; + if (proxyMinerName in proxyMiners) proxyMiners[proxyMinerName].hashes += job.norm_diff; + + const time_now = Date.now(); + let wallet_key = miner.wallet_key + blockTemplate.port; + + if (!(wallet_key in walletAcc)) { + walletAcc[wallet_key] = {}; + walletWorkerCount[wallet_key] = 0; + is_walletAccFinalizer[wallet_key] = false; + } + + const db_job_height = global.config.daemon.port == blockTemplate.port ? blockTemplate.height : anchorBlockHeight; + + let wallet = walletAcc[wallet_key]; + const worker_name = miner.identifier in wallet || walletWorkerCount[wallet_key] < 50 ? miner.identifier : "all_other_workers"; + + if (!(worker_name in wallet)) { + if (worker_name !== "all_other_workers") ++ walletWorkerCount[wallet_key]; + debug("!!! " + wallet_key + ": adding new worker " + worker_name + " (num " + walletWorkerCount[wallet_key] + ")"); + wallet[worker_name] = {}; + let worker = wallet[worker_name]; + worker.height = db_job_height; + worker.difficulty = blockTemplate.difficulty; + worker.time = time_now; + worker.acc = 0; + worker.acc2 = 0; + worker.share_num = 0; + } + + let worker = wallet[worker_name]; + + let height = worker.height; + let difficulty = worker.difficulty; + let acc = worker.acc; + let acc2 = worker.acc2; + let share_num = worker.share_num; + + if (time_now - worker.time > global.config.pool.shareAccTime*1000 || acc >= 100000000) { + if (acc != 0) { + debug("!!! " + wallet_key + " / " + worker_name + ": storing share " + height + " " + difficulty + " " + time_now + " " + acc); + storeShareDiv(miner, acc, acc2, share_num, worker_name, blockTemplate.port, height, difficulty, false, isTrustedShare); } - if (this.newDiff < global.config.pool.minDifficulty) { - this.newDiff = global.config.pool.minDifficulty; + + worker.height = db_job_height; + worker.difficulty = blockTemplate.difficulty; + worker.time = time_now; + worker.acc = job.rewarded_difficulty; + worker.acc2 = job.rewarded_difficulty2; + worker.share_num = 1; + + } else { + worker.acc += job.rewarded_difficulty; + worker.acc2 += job.rewarded_difficulty2; + ++ worker.share_num; + } + + debug("!!! " + wallet_key + " / " + worker_name + ": accumulating share " + db_job_height + " " + blockTemplate.difficulty + " " + worker.time + " " + worker.acc + " (+" + job.rewarded_difficulty + ")"); + + if (is_walletAccFinalizer[wallet_key] === false) { + is_walletAccFinalizer[wallet_key] = true; + setTimeout(walletAccFinalizer, global.config.pool.shareAccTime*1000, wallet_key, miner, blockTemplate.port); + } + + if (isTrustedShare) { + process.send({type: 'trustedShare'}); + debug(threadName + "Accepted trusted share at difficulty: " + job.difficulty + "/" + job.rewarded_difficulty + " from: " + miner.logString); + } else { + process.send({type: 'normalShare'}); + debug(threadName + "Accepted valid share at difficulty: " + job.difficulty + "/" + job.rewarded_difficulty + " from: " + miner.logString); + } + if (activeBlockTemplates[job.coin].idHash !== blockTemplate.idHash) { + process.send({type: 'outdatedShare'}); + } + +} + +function getShareBuffer(miner, job, blockTemplate, params) { + try { + let template = Buffer.alloc(blockTemplate.buffer.length); + blockTemplate.buffer.copy(template); + template.writeUInt32BE(job.extraNonce, blockTemplate.reserved_offset); + if (miner.proxy) { + template.writeUInt32BE(params.poolNonce, job.clientPoolLocation); + template.writeUInt32BE(params.workerNonce, job.clientNonceLocation); } - debug(threadName + "Difficulty change to: " + this.newDiff + " For: " + this.logString); - if (this.hashes > 0) { - debug(threadName + "Hashes: " + this.hashes + " in: " + Math.floor((Date.now() - this.connectTime) / 1000) + " seconds gives: " + - Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) + " hashes/second or: " + - Math.floor(this.hashes / (Math.floor((Date.now() - this.connectTime) / 1000))) * global.config.pool.targetTime + " difficulty versus: " + this.newDiff); + return global.coinFuncs.constructNewBlob(template, params, blockTemplate.port); + } catch (e) { + const err_str = "Can't constructNewBlob of " + blockTemplate.port + " port with " + JSON.stringify(params) + " params from " + miner.logString + ": " + e; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't constructNewBlob", err_str); + return null; + } +} + + +function invalid_share(miner) { + process.send({type: 'invalidShare'}); + miner.sendSameCoinJob(); + walletTrust[miner.payout] = 0; + return false; +} + +function submit_block(miner, job, blockTemplate, blockData, resultBuff, isTrustedShare, isParentBlock, isRetrySubmitBlock, submit_blockCB) { + let reply_fn = function (rpcResult, rpcStatus) { + const blockDataStr = Buffer.isBuffer(blockData) ? blockData.toString('hex') : JSON.stringify(blockData); + const blob_type_num = global.coinFuncs.portBlobType(blockTemplate.port, blockTemplate.block_version); + if (rpcResult && (rpcResult.error || rpcResult.result === "high-hash" || rpcResult.result === "bad-txnmrklroot" || rpcResult.result === "bad-cbtx-mnmerkleroot")) { // did not manage to submit a block + let isNotifyAdmin = true; + if (isParentBlock && isTrustedShare) { + const convertedBlob = global.coinFuncs.convertBlob(blockData, blockTemplate.port); + const buff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate); + if (!buff.equals(resultBuff)) isNotifyAdmin = false; + } + + console.error(threadName + "Error submitting " + blockTemplate.coin + " (port " + blockTemplate.port + ") block at height " + + blockTemplate.height + " (active block template height: " + activeBlockTemplates[blockTemplate.coin].height + ") from " + + miner.logString + ", isTrustedShare: " + isTrustedShare + ", valid: " + isNotifyAdmin + ", rpcStatus: " + rpcStatus + + ", error: " + JSON.stringify(rpcResult) + ", block hex: \n" + blockDataStr + ); + + if (isNotifyAdmin) setTimeout(function() { // only alert if block height is not changed in the nearest time + global.coinFuncs.getPortLastBlockHeader(blockTemplate.port, function(err, body) { + if (err !== null) { + console.error("Last block header request failed for " + blockTemplate.port + " port!"); + return; + } + if (blockTemplate.height == body.height + 1) global.support.sendEmail(global.config.general.adminEmail, + "FYI: Can't submit " + blockTemplate.coin + " block to deamon on " + blockTemplate.port + " port", + "The pool server: " + global.config.hostname + " can't submit block to deamon on " + blockTemplate.port + " port\n" + + "Input: " + blockDataStr + "\n" + + threadName + "Error submitting " + blockTemplate.coin + " block at " + blockTemplate.height + " height from " + miner.logString + + ", isTrustedShare: " + isTrustedShare + " error ): " + JSON.stringify(rpcResult) + ); + }); + }, 2*1000); + + if (global.config.pool.trustedMiners) { + debug(threadName + "Share trust broken by " + miner.logString); + miner.trust.trust = 0; + walletTrust[miner.payout] = 0; + } + + if (submit_blockCB) submit_blockCB(false); + + // Success! Submitted a block without an issue. + } else if ( rpcResult && ( + ( typeof(rpcResult.result) !== 'undefined' ) || + ( rpcResult.response !== 'rejected' && global.coinFuncs.blobTypeErg(blob_type_num) ) || // ERG + ( typeof rpcResult === 'string' && rpcStatus == 202 && blockTemplate.port == 11898 ) // TRTL + ) + ) { + + const get_block_id = function(cb) { + if (global.coinFuncs.blobTypeDero(blob_type_num)) { + return cb(rpcResult.result.blid); + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + return cb(resultBuff.toString('hex')); + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + setTimeout(global.coinFuncs.getPortBlockHeaderByID, 10*1000, blockTemplate.port, blockTemplate.height, function(err, body) { + if (err === null && body.powSolutions.pk === blockTemplate.hash2) return cb(body.id); + return cb("0000000000000000000000000000000000000000000000000000000000000000"); + }); + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + setTimeout(global.coinFuncs.ethBlockFind, 30*1000, blockTemplate.port, blockData[0], function(block_hash) { + return cb(block_hash ? block_hash.substr(2) : "0000000000000000000000000000000000000000000000000000000000000000"); + }); + } else { + return cb(global.coinFuncs.getBlockID(blockData, blockTemplate.port).toString('hex')); + } + }; + + get_block_id(function(newBlockHash) { + console.log(threadName + "New " + blockTemplate.coin + " (port " + blockTemplate.port + ") block " + newBlockHash + " found at height " + blockTemplate.height + " by " + miner.logString + + ", isTrustedShare: " + isTrustedShare + " - submit result: " + JSON.stringify(rpcResult) + + ", block hex: \n" + blockDataStr + ); + + const time_now = Date.now(); + if (global.config.daemon.port == blockTemplate.port) { + global.database.storeBlock(blockTemplate.height, global.protos.Block.encode({ + hash: newBlockHash, + difficulty: blockTemplate.difficulty, + shares: 0, + timestamp: time_now, + poolType: miner.poolTypeEnum, + unlocked: false, + valid: true + })); + } else { + global.database.storeAltBlock(Math.floor(time_now / 1000), global.protos.AltBlock.encode({ + hash: newBlockHash, + difficulty: blockTemplate.difficulty, + shares: 0, + timestamp: time_now, + poolType: miner.poolTypeEnum, + unlocked: false, + valid: true, + port: blockTemplate.port, + height: blockTemplate.height, + anchor_height: anchorBlockHeight + })); + } + + if (submit_blockCB) submit_blockCB(true); + }); + + } else { // something not expected happened + if (isRetrySubmitBlock) { + console.error(threadName + "Unknown error submitting " + blockTemplate.coin + " (port " + blockTemplate.port + ") block at height " + + blockTemplate.height + " (active block template height: " + activeBlockTemplates[blockTemplate.coin].height + ") from " + + miner.logString + ", isTrustedShare: " + isTrustedShare + ", rpcStatus: " + rpcStatus + ", error (" + (typeof rpcResult) + "): " + JSON.stringify(rpcResult) + + ", block hex: \n" + blockDataStr + ); + setTimeout(submit_block, 500, miner, job, blockTemplate, blockData, resultBuff, isTrustedShare, isParentBlock, false, submit_blockCB); + } else { + // RPC bombed out massively. + console.error(threadName + "RPC Error. Please check logs for details"); + global.support.sendEmail(global.config.general.adminEmail, + "FYI: Can't submit block to deamon on " + blockTemplate.port + " port", + "Input: " + blockDataStr + "\n" + + "The pool server: " + global.config.hostname + " can't submit block to deamon on " + blockTemplate.port + " port\n" + + "RPC Error. Please check logs for details" + ); + if (submit_blockCB) submit_blockCB(false); + } } - this.sendNewJob(); }; - this.checkBan = function (validShare) { - if (!global.config.pool.banEnabled) { - return; + if (blockTemplate.port == 11898) { + global.support.rpcPortDaemon2(blockTemplate.port, "block", blockData.toString('hex'), reply_fn); + } else if (global.coinFuncs.blobTypeRvn(job.blob_type_num) || global.coinFuncs.blobTypeRtm(job.blob_type_num) || global.coinFuncs.blobTypeKcn(job.blob_type_num)) { + global.support.rpcPortDaemon2(blockTemplate.port, "", { method: "submitblock", params: [ blockData.toString('hex') ] }, reply_fn); + } else if (global.coinFuncs.blobTypeEth(job.blob_type_num)) { + global.support.rpcPortDaemon2(blockTemplate.port, "", { method: "eth_submitWork", params: blockData, jsonrpc: "2.0", id: 0 }, reply_fn); + } else if (global.coinFuncs.blobTypeErg(job.blob_type_num)) { + global.support.rpcPortDaemon2(blockTemplate.port, "mining/solution", {"n": blockData}, reply_fn); + } else if (global.coinFuncs.blobTypeDero(job.blob_type_num)) { + global.support.rpcPortDaemon(blockTemplate.port, "submitblock", [ blockTemplate.blocktemplate_blob, blockData.toString('hex') ], reply_fn); + } else { + global.support.rpcPortDaemon(blockTemplate.port, "submitblock", [ blockData.toString('hex') ], reply_fn); + } +} + +// wallets that need extra share verification +let extra_wallet_verify = {}; +let extra_verify_wallet_hashes = []; + +function is_safe_to_trust(reward_diff, miner_wallet, miner_trust) { + const reward_diff2 = reward_diff * global.config.pool.trustThreshold; + return reward_diff < 400000 && miner_trust != 0 && ( + ( miner_wallet in walletTrust && + reward_diff2 * global.config.pool.trustThreshold < walletTrust[miner_wallet] && + crypto.randomBytes(1).readUIntBE(0, 1) > global.config.pool.trustMin + ) || ( + reward_diff2 < miner_trust && + crypto.randomBytes(1).readUIntBE(0, 1) > Math.max(256 - miner_trust / reward_diff / 2, global.config.pool.trustMin) + ) + ); +} + +function hashBuffDiff(hash) { // bignum as result + return baseDiff.div(bignum.fromBuffer(hash, {endian: 'little', size: 32})); +} + +function hashRavenBuffDiff(hash) { // float as result + return baseRavenDiff / bignum.fromBuffer(hash).toNumber(); +} + +function hashEthBuffDiff(hash) { // bignum as result + return baseDiff.div(bignum.fromBuffer(hash)); +} + +// will work for numbers and bignum +function ge(l, r) { + if (typeof l === 'object') return l.ge(r); + if (typeof r === 'object') return !r.lt(l); + return l >= r; +} + +function report_miner_share(miner, job) { + const time_now = Date.now(); + if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { + console.error(threadName + "Bad " + job.coin + " coin share from miner (diff " + job.difficulty + ") " + miner.logString); + lastMinerLogTime[miner.payout] = time_now; + } +} + +function processShare(miner, job, blockTemplate, params, processShareCB) { + const port = blockTemplate.port; + const blob_type_num = job.blob_type_num; + + if (miner.payout in minerWallets) minerWallets[miner.payout].hashes += job.norm_diff; + walletLastSeeTime[miner.payout] = Date.now(); + + let shareThrottled = function(processShareCB) { + if (miner.payout in minerWallets) { + const last_ver_shares = ++minerWallets[miner.payout].last_ver_shares; + const threshold = global.config.pool.minerThrottleSharePerSec * global.config.pool.minerThrottleShareWindow; + if (last_ver_shares > threshold) { + if (last_ver_shares == threshold) { + console.error(threadName + "Throttled down miner share (diff " + job.rewarded_difficulty2 + ") submission from " + miner.logString); + } else if (job.rewarded_difficulty2 >= 10000000 && last_ver_shares > 10 * threshold) { // too much will invalidate share + console.error(threadName + "Throttled down miner share as invalid (diff " + job.rewarded_difficulty2 + ") submission from " + miner.logString); + invalid_share(miner); + processShareCB(false); // invalid share + return true; + } + process.send({type: 'throttledShare'}); + if (addProxyMiner(miner)) { + const proxyMinerName = miner.payout; // + ":" + miner.identifier; + proxyMiners[proxyMinerName].hashes += job.norm_diff; + adjustMinerDiff(miner); + } + processShareCB(null); // throttle share + return true; + } } + return false; + } - // Valid stats are stored by the pool. - if (validShare) { - this.validShares += 1; + let verifyShare = function(verifyShareCB) { + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + const blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData === null) return processShareCB(invalid_share(miner)); + const header = Buffer.concat([global.coinFuncs.convertBlob(blockData, port), bignum(params.nonce, 10).toBuffer({endian: 'big', size: 4})]); + if (global.coinFuncs.c29(header, params.pow, port)) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); + } + const resultBuff = global.coinFuncs.c29_cycle_hash(params.pow, blob_type_num); + return verifyShareCB(hashBuffDiff(resultBuff), resultBuff, blockData, false, true); + + } else if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + const blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData === null) return processShareCB(invalid_share(miner)); + const convertedBlob = global.coinFuncs.convertBlob(blockData, port); + if (params.header_hash !== convertedBlob.toString("hex")) { + console.error("Wrong header hash:" + params.header_hash + " " + convertedBlob.toString("hex")); + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); + } + const resultBuff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate, params.nonce, params.mixhash); + return verifyShareCB(hashRavenBuffDiff(resultBuff), resultBuff, blockData, false, true); + + } else if (global.coinFuncs.blobTypeEth(blob_type_num)) { + if (shareThrottled(processShareCB)) return; + const hashes = global.coinFuncs.slowHashBuff(Buffer.from(blockTemplate.hash, 'hex'), blockTemplate, params.nonce); + const resultBuff = hashes[0]; + const blockData = [ "0x" + params.nonce, "0x" + blockTemplate.hash, "0x" + hashes[1].toString('hex') ]; + return verifyShareCB(hashEthBuffDiff(resultBuff), resultBuff, blockData, false, true); + + } else if (global.coinFuncs.blobTypeErg(blob_type_num)) { + if (shareThrottled(processShareCB)) return; + const coinbaseBuffer = Buffer.concat([Buffer.from(blockTemplate.hash, 'hex'), Buffer.from(params.nonce, 'hex')]); + const hashes = global.coinFuncs.slowHashBuff(coinbaseBuffer, blockTemplate); + return verifyShareCB(hashEthBuffDiff(hashes[1]), null, params.nonce, false, true); + } + + const resultHash = params.result; + let resultBuff; + try { + resultBuff = Buffer.from(resultHash, 'hex'); + } catch(e) { + return processShareCB(invalid_share(miner)); + } + const hashDiff = hashBuffDiff(resultBuff); + + if ( global.config.pool.trustedMiners && + is_safe_to_trust(job.rewarded_difficulty2, miner.payout, miner.trust.trust) && + miner.trust.check_height !== job.height + ) { + let blockData = null; + if (miner.payout in extra_wallet_verify) { + blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData !== null) { + const convertedBlob = global.coinFuncs.convertBlob(blockData, port); + global.coinFuncs.slowHashAsync(convertedBlob, blockTemplate, miner.payout, function(hash) { + if (hash === null || hash === false) { + console.error(threadName + "[EXTRA CHECK] Can't verify share remotely!"); + } else if (hash !== resultHash) { + console.error(threadName + miner.logString + " [EXTRA CHECK] INVALID SHARE OF " + job.rewarded_difficulty2 + " REWARD HASHES"); + } else { + extra_verify_wallet_hashes.push(miner.payout + " " + convertedBlob.toString('hex') + " " + resultHash + " " + global.coinFuncs.algoShortTypeStr(port) + " " + blockTemplate.height + " " + blockTemplate.seed_hash); + } + }); + } else { + console.error(threadName + miner.logString + " [EXTRA CHECK] CAN'T MAKE SHARE BUFFER"); + } + } + if (miner.lastSlowHashAsyncDelay) { + setTimeout(function() { return verifyShareCB(hashDiff, resultBuff, blockData, true, true); }, miner.lastSlowHashAsyncDelay); + debug("[MINER] Delay " + miner.lastSlowHashAsyncDelay); + } else { + return verifyShareCB(hashDiff, resultBuff, blockData, true, true); + } + + } else { // verify share + if (miner.debugMiner) console.log(threadName + miner.logString + " [WALLET DEBUG] verify share"); + if (shareThrottled(processShareCB)) return; + const blockData = getShareBuffer(miner, job, blockTemplate, params); + if (blockData === null) return processShareCB(invalid_share(miner)); + const convertedBlob = global.coinFuncs.convertBlob(blockData, port); + + const isBlockDiffMatched = ge(hashDiff, blockTemplate.difficulty); + if (isBlockDiffMatched) { + if (miner.validShares || (miner.payout in minerWallets && minerWallets[miner.payout].hashes)) { + submit_block(miner, job, blockTemplate, blockData, resultBuff, true, true, true, function(block_submit_result) { + if (!block_submit_result) { + const buff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate); + if (!buff.equals(resultBuff)) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); + } + } + walletTrust[miner.payout] += job.rewarded_difficulty2; + return verifyShareCB(hashDiff, resultBuff, blockData, false, false); + }); + } else { + const buff = global.coinFuncs.slowHashBuff(convertedBlob, blockTemplate); + if (!buff.equals(resultBuff)) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); + } + walletTrust[miner.payout] += job.rewarded_difficulty2; + return verifyShareCB(hashDiff, resultBuff, blockData, false, true); + } + } else { + const time_now = Date.now(); + global.coinFuncs.slowHashAsync(convertedBlob, blockTemplate, miner.payout, function(hash) { + if (hash === null) { + return processShareCB(null); + //} else if (hash === false) { + // console.error(threadName + "Processed share locally instead of remotely!"); + // hash = global.coinFuncs.slowHash(convertedBlob, blockTemplate); + } + if (hash !== resultHash) { + report_miner_share(miner, job); + return processShareCB(invalid_share(miner)); + } + miner.lastSlowHashAsyncDelay = Date.now() - time_now; + if (miner.lastSlowHashAsyncDelay > 1000) miner.lastSlowHashAsyncDelay = 1000; + walletTrust[miner.payout] += job.rewarded_difficulty2; + return verifyShareCB(hashDiff, resultBuff, blockData, false, false); + }); + } + } + }; + + verifyShare(function(hashDiff, resultBuff, blockData, isTrustedShare, isNeedCheckBlockDiff) { + if (isNeedCheckBlockDiff && ge(hashDiff, blockTemplate.difficulty)) { // Submit block to the RPC Daemon. + if (!blockData) { + blockData = getShareBuffer(miner, job, blockTemplate, params); + if (!blockData) return processShareCB(invalid_share(miner)); + } + submit_block(miner, job, blockTemplate, blockData, resultBuff, isTrustedShare, true, true); + } + + const is_mm = "child_template" in blockTemplate; + if (is_mm && ge(hashDiff, blockTemplate.child_template.difficulty)) { // Submit child block to the RPC Daemon. + if (!blockData) { + blockData = getShareBuffer(miner, job, blockTemplate, params); + if (!blockData) return processShareCB(invalid_share(miner)); + } + // need to properly restore child template buffer here since it went via message string and was restored not correctly + blockTemplate.child_template_buffer = Buffer.from(blockTemplate.child_template_buffer); + let shareBuffer2 = null; + try { + shareBuffer2 = global.coinFuncs.constructMMChildBlockBlob(blockData, port, blockTemplate.child_template_buffer); + } catch (e) { + const err_str = "Can't construct_mm_child_block_blob with " + blockData.toString('hex') + " parent block and " + blockTemplate.child_template_buffer.toString('hex') + " child block share buffers from " + miner.logString + ": " + e; + console.error(err_str); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Can't construct_mm_child_block_blob", err_str); + return processShareCB(invalid_share(miner)); + } + if (shareBuffer2 === null) return processShareCB(invalid_share(miner)); + submit_block(miner, job, blockTemplate.child_template, shareBuffer2, resultBuff, isTrustedShare, false, true); + } + + if (!ge(hashDiff, job.difficulty)) { + let time_now = Date.now(); + if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { + console.warn(threadName + "Rejected low diff (" + hashDiff + " < " + job.difficulty + ") share from miner " + miner.logString); + lastMinerLogTime[miner.payout] = time_now; + } + return processShareCB(invalid_share(miner)); + } else { - this.invalidShares += 1; + recordShareData(miner, job, isTrustedShare, blockTemplate); + // record child proc share for rewarded_difficulty effort calcs status but with 0 rewards (all included in parent share) + if (is_mm) { + job.rewarded_difficulty2 = 0; + recordShareData(miner, job, isTrustedShare, blockTemplate.child_template); + } + return processShareCB(true); } - if (this.validShares + this.invalidShares >= global.config.pool.banThreshold) { - if (this.invalidShares / this.validShares >= global.config.pool.banPercent / 100) { - delete activeMiners[this.id]; - process.send({type: 'banIP', data: this.ipAddress}); + }); +} + +// Message times for different miner addresses +let lastMinerLogTime = {}; +// Miner notification times +let lastMinerNotifyTime = {}; + +function get_miner_notification(payout) { + if (payout in notifyAddresses) return notifyAddresses[payout]; + return false; +} + +function handleMinerData(socket, id, method, params, ip, portData, sendReply, sendReplyFinal, pushMessage) { + switch (method) { + case 'mining.authorize': // Eth/Raven/Erg only + if (!params || !(params instanceof Array)) { + sendReplyFinal("No array params specified"); + return; + } + params = { + login: params[0], + pass: params[1], + agent: socket.eth_agent ? socket.eth_agent : "[generic_ethminer]", + algo: [ "kawpow" ], + "algo-perf": { "kawpow": 1 }, + }; + // continue to normal login + + case 'login': { // Grin and default + if (ip in bannedTmpIPs) { + sendReplyFinal("New connections from this IP address are temporarily suspended from mining (10 minutes max)"); + return; + } + if (!params) { + process.send({type: 'banIP', data: ip}); + sendReplyFinal("No params specified"); + return; + } + if (!params.login) { + process.send({type: 'banIP', data: ip}); + sendReplyFinal("No login specified"); + return; + } + if (socket.miner_id) { + process.send({type: 'banIP', data: ip}); + sendReplyFinal("No double login is allowed"); + return; } - else { - this.invalidShares = 0; - this.validShares = 0; + if (!params.pass) params.pass = "x"; + const difficulty = portData.difficulty; + const minerId = get_new_id(); + let miner = new Miner( + minerId, params.login, params.pass, params.rigid, ip, difficulty, pushMessage, 1, portData.portType, portData.port, params.agent, + params.algo, params["algo-perf"], params["algo-min-time"] + ); + if (miner.debugMiner) socket.debugMiner = 1; //console.log(threadName + miner.logString + " [WALLET DEBUG] " + method + ": " + JSON.stringify(params)); + if (method === 'mining.authorize') { + const new_id = socket.eth_extranonce_id ? socket.eth_extranonce_id : get_new_eth_extranonce_id(); + if (new_id !== null) { + socket.eth_extranonce_id = new_id; + miner.eth_extranonce = eth_extranonce(new_id); + } else { + miner.valid_miner = false; + miner.error = "Not enough extranoces. Switch to other pool node."; + } } - } - }; - - if (protoVersion === 1) { - this.getTargetHex = function () { - if (this.newDiff) { - this.difficulty = this.newDiff; - this.newDiff = null; - } - let padded = new Buffer(32); - padded.fill(0); - let diffBuff = baseDiff.div(this.difficulty).toBuffer(); - diffBuff.copy(padded, 32 - diffBuff.length); - - let buff = padded.slice(0, 4); - let buffArray = buff.toByteArray().reverse(); - let buffReversed = new Buffer(buffArray); - this.target = buffReversed.readUInt32BE(0); - return buffReversed.toString('hex'); - }; - this.getJob = function () { - - if (this.lastBlockHeight === activeBlockTemplate.height && activeBlockTemplate.idHash === this.validJobs.get(0).blockHash && !this.newDiff && this.cachedJob !== null) { - return this.cachedJob; + if (params.agent && process.env['WORKER_ID'] == 1) minerAgents[params.agent] = 1; + let time_now = Date.now(); + if (!miner.valid_miner) { + if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 10*60*1000) { + console.log("Invalid miner " + miner.logString + " [" + miner.email + "], disconnecting due to: " + miner.error); + lastMinerLogTime[miner.payout] = time_now; + } + return sendReplyFinal(miner.error, miner.delay_reply); } - if (!this.proxy) { - let blob = activeBlockTemplate.nextBlob(); - let target = this.getTargetHex(); - this.lastBlockHeight = activeBlockTemplate.height; + const miner_agent_notification = !global.coinFuncs.algoMainCheck(miner.algos) && global.coinFuncs.algoPrevMainCheck(miner.algos) ? + global.coinFuncs.get_miner_agent_warning_notification(params.agent) : false; + const miner_notification = miner_agent_notification ? miner_agent_notification : get_miner_notification(miner.payout); + if (miner_notification) { + if (!(miner.payout in lastMinerNotifyTime) || time_now - lastMinerNotifyTime[miner.payout] > 60*60*1000) { + lastMinerNotifyTime[miner.payout] = time_now; + console.error("Sent notification to " + miner.logString + ": " + miner_notification); + return sendReplyFinal(miner_notification + " (miner will connect after several attempts)"); + } + } + if (!miner.proxy) { + let proxyMinerName = miner.payout; // + ":" + miner.identifier; + if ((params.agent && params.agent.includes('proxy')) || (proxyMinerName in proxyMiners)) { + if (!addProxyMiner(miner)) { + return sendReplyFinal("Temporary (one hour max) mining ban since you connected too many workers. Please use proxy (https://github.com/MoneroOcean/xmrig-proxy)", 600); + } + if (proxyMiners[proxyMinerName].hashes) adjustMinerDiff(miner); + } else { + if (!(miner.payout in minerWallets)) { + minerWallets[miner.payout] = {}; + minerWallets[miner.payout].connectTime = Date.now(); + minerWallets[miner.payout].count = 1; + minerWallets[miner.payout].hashes = 0; + minerWallets[miner.payout].last_ver_shares = 0; + } else { + if (++ minerWallets[miner.payout].count > global.config.pool.workerMax) { + bannedBigTmpWallets[miner.payout] = 1; + return sendReplyFinal("Temporary (one hour max) ban on new miner connections since you connected too many workers. Please use proxy (https://github.com/MoneroOcean/xmrig-proxy)", 600); + } + } + } + } - let newJob = { - id: crypto.pseudoRandomBytes(21).toString('base64'), - extraNonce: activeBlockTemplate.extraNonce, - height: activeBlockTemplate.height, - difficulty: this.difficulty, - diffHex: this.diffHex, - submissions: [], - blockHash: activeBlockTemplate.idHash - }; + socket.miner_id = minerId; + activeMiners.set(minerId, miner); + + if (id === "Stratum") { // if grin miner is connected directly to the pool + sendReply(null, "ok"); + miner.protocol = "grin"; + } else if (method === 'mining.authorize') { // if raven/eth miner is connected directly to the pool + sendReply(null, true); + miner.protocol = "eth"; // technically equivalent to "default" + miner.sendBestCoinJob(); + } else { // if meta-miner or xmrig or something else connected + const coin = miner.selectBestCoin(); + if (coin !== false) { + const params = getCoinJobParams(coin); + const blob_type_num = global.coinFuncs.portBlobType(global.coinFuncs.COIN2PORT(coin)); + if ( global.coinFuncs.blobTypeRvn(blob_type_num) || + global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num) + ) { // xmrig specifics + const new_id = socket.eth_extranonce_id ? socket.eth_extranonce_id : get_new_eth_extranonce_id(); + if (new_id !== null) { + socket.eth_extranonce_id = new_id; + miner.eth_extranonce = eth_extranonce(new_id); + sendReply(null, { id: minerId, algo: params.algo_name, extra_nonce: miner.eth_extranonce }); + miner.sendCoinJob(coin, params); + } else { + sendReplyFinal("Not enough extranoces. Switch to other pool node."); + } + } else { + sendReply(null, { id: minerId, job: miner.getCoinJob(coin, params), status: 'OK' }); + } + } else { + sendReplyFinal("No block template yet. Please wait."); + } + miner.protocol = "default"; + } + break; + } - this.validJobs.enq(newJob); - this.cachedJob = { - blob: blob, - job_id: newJob.id, - target: target, - id: this.id - }; + case 'mining.subscribe': { // Raven/Eth/Erg only + if (params && (params instanceof Array) && params.length >= 1) socket.eth_agent = params[0]; + const new_id = socket.eth_extranonce_id ? socket.eth_extranonce_id : get_new_eth_extranonce_id(); + if (new_id !== null) { + socket.eth_extranonce_id = new_id; + // extranonce is not really needed for Raven (extraonce is specificed as part of coinbase tx) + sendReply(null, [ [ "mining.notify", get_new_id(), "EthereumStratum/1.0.0" ], eth_extranonce(new_id), 6 ]); } else { - let blob = activeBlockTemplate.nextBlobWithChildNonce(); - if (this.newDiff) { - this.difficulty = this.newDiff; - this.newDiff = null; - } - this.lastBlockHeight = activeBlockTemplate.height; - - let newJob = { - id: crypto.pseudoRandomBytes(21).toString('base64'), - extraNonce: activeBlockTemplate.extraNonce, - height: activeBlockTemplate.height, - difficulty: this.difficulty, - diffHex: this.diffHex, - clientPoolLocation: activeBlockTemplate.clientPoolLocation, - clientNonceLocation: activeBlockTemplate.clientNonceLocation, - submissions: [] - }; - this.validJobs.enq(newJob); - this.cachedJob = { - blocktemplate_blob: blob, - difficulty: activeBlockTemplate.difficulty, - height: activeBlockTemplate.height, - reserved_offset: activeBlockTemplate.reserveOffset, - client_nonce_offset: activeBlockTemplate.clientNonceLocation, - client_pool_offset: activeBlockTemplate.clientPoolLocation, - target_diff: this.difficulty, - target_diff_hex: this.diffHex, - job_id: newJob.id, - id: this.id - }; + sendReplyFinal("Not enough extranoces. Switch to other pool node."); } - return this.cachedJob; - }; + break; + } - this.sendNewJob = function() { - let job = this.getJob(); - let tempJob = this.sentJobs.toarray().filter(function (intJob) { - return intJob.id === job.job_id; - })[0]; + case 'mining.extranonce.subscribe': { // Raven/Eth only + sendReply(null, true); + break; + } - if (tempJob) { - console.error(`Tried sending a duped job to: ${this.address}, stopped by Snipa!`); + case 'getjobtemplate': { // grin-mode miner only + const minerId = socket.miner_id ? socket.miner_id : ""; + let miner = activeMiners.get(minerId); + if (!miner) { + sendReplyFinal("Unauthenticated"); return; } - this.sentJobs.enq(job); - return this.messageSender('job', job); - }; - } -} - -function recordShareData(miner, job, shareDiff, blockCandidate, hashHex, shareType, blockTemplate) { - miner.hashes += job.difficulty; - global.database.storeShare(job.height, global.protos.Share.encode({ - shares: job.difficulty, - paymentAddress: miner.address, - paymentID: miner.paymentID, - foundBlock: blockCandidate, - trustedShare: shareType, - poolType: miner.poolTypeEnum, - poolID: global.config.pool_id, - blockDiff: activeBlockTemplate.difficulty, - bitcoin: miner.bitcoin, - blockHeight: job.height, - timestamp: Date.now(), - identifier: miner.identifier - })); - if (blockCandidate) { - global.database.storeBlock(job.height, global.protos.Block.encode({ - hash: hashHex, - difficulty: blockTemplate.difficulty, - shares: 0, - timestamp: Date.now(), - poolType: miner.poolTypeEnum, - unlocked: false, - valid: true - })); - } - if (shareType) { - process.send({type: 'trustedShare'}); - debug(threadName + "Accepted trusted share at difficulty: " + job.difficulty + "/" + shareDiff + " from: " + miner.logString); - } else { - process.send({type: 'normalShare'}); - debug(threadName + "Accepted valid share at difficulty: " + job.difficulty + "/" + shareDiff + " from: " + miner.logString); - } - -} + miner.heartbeat(); + sendReply(null, miner.getBestCoinJob()); + break; + } -function processShare(miner, job, blockTemplate, params) { - let nonce = params.nonce; - let resultHash = params.result; - let template = new Buffer(blockTemplate.buffer.length); - if (!miner.proxy) { - blockTemplate.buffer.copy(template); - template.writeUInt32BE(job.extraNonce, blockTemplate.reserveOffset); - } else { - blockTemplate.buffer.copy(template); - template.writeUInt32BE(job.extraNonce, blockTemplate.reserveOffset); - template.writeUInt32BE(params.poolNonce, job.clientPoolLocation); - template.writeUInt32BE(params.workerNonce, job.clientNonceLocation); - } - let shareBuffer = global.coinFuncs.constructNewBlob(template, new Buffer(nonce, 'hex')); - - let convertedBlob; - let hash; - let shareType; - - if (global.config.pool.trustedMiners && miner.trust.threshold <= 0 && miner.trust.penalty <= 0 && - crypto.randomBytes(1).readUIntBE(0, 1) > miner.trust.probability) { - hash = new Buffer(resultHash, 'hex'); - shareType = true; - } - else { - convertedBlob = global.coinFuncs.convertBlob(shareBuffer); - hash = global.coinFuncs.cryptoNight(convertedBlob); - shareType = false; - } - if (hash.toString('hex') !== resultHash) { - console.error(threadName + "Bad share from miner " + miner.logString); - process.send({type: 'invalidShare'}); - if (miner.incremented === false) { - miner.newDiff = miner.difficulty + 1; - miner.incremented = true; - } else { - miner.newDiff = miner.difficulty - 1; - miner.incremented = false; + case 'getjob': { + if (!params) { + sendReplyFinal("No params specified"); + return; + } + let miner = activeMiners.get(params.id); + if (!miner) { + sendReplyFinal("Unauthenticated"); + return; + } + miner.heartbeat(); + if (params.algo && params.algo instanceof Array && params["algo-perf"] && params["algo-perf"] instanceof Object) { + const status = miner.setAlgos(params.algo, params["algo-perf"], params["algo-min-time"]); + if (status != "") { + sendReply(status); + return; + } + } + sendReply(null, miner.getBestCoinJob()); + break; } - miner.sendNewJob(); - return false; - } - let hashArray = hash.toByteArray().reverse(); - let hashNum = bignum.fromBuffer(new Buffer(hashArray)); - let hashDiff = baseDiff.div(hashNum); - - - if (hashDiff.ge(blockTemplate.difficulty)) { - // Submit block to the RPC Daemon. - // Todo: Implement within the coins/.js file. - global.support.rpcDaemon('submitblock', [shareBuffer.toString('hex')], function (rpcResult) { - if (rpcResult.error) { - // Did not manage to submit a block. Log and continue on. - console.error(threadName + "Error submitting block at height " + job.height + " from " + miner.logString + ", share type: " + shareType + " error: " + JSON.stringify(rpcResult.error)); - recordShareData(miner, job, hashDiff.toString(), false, null, shareType); - // Error on submit, so we'll submit a sanity check for good measure. - templateUpdate(); - } else if (rpcResult) { - //Success! Submitted a block without an issue. - let blockFastHash = global.coinFuncs.getBlockID(shareBuffer).toString('hex'); - console.log(threadName + "Block " + blockFastHash.substr(0, 6) + " found at height " + job.height + " by " + miner.logString + - ", share type: " + shareType + " - submit result: " + JSON.stringify(rpcResult.result)); - recordShareData(miner, job, hashDiff.toString(), true, blockFastHash, shareType, blockTemplate); - templateUpdate(); - } else { - // RPC bombed out massively. - console.error(threadName + "RPC Error. Please check logs for details"); + case 'mining.submit': + if (!params || !(params instanceof Array)) { + sendReply("No array params specified"); + return; } - }); - } - else if (hashDiff.lt(job.difficulty)) { - process.send({type: 'invalidShare'}); - console.warn(threadName + "Rejected low diff share of " + hashDiff.toString() + " from: " + miner.address + " ID: " + - miner.identifier + " IP: " + miner.ipAddress); - return false; - } - else { - recordShareData(miner, job, hashDiff.toString(), false, null, shareType); - } - return true; -} -function handleMinerData(method, params, ip, portData, sendReply, pushMessage) { - let miner = activeMiners[params.id]; - // Check for ban here, so preconnected attackers can't continue to screw you - if (bannedIPs.indexOf(ip) !== -1) { - // Handle IP ban off clip. - sendReply("IP Address currently banned"); - return; - } - switch (method) { - case 'login': - if (!params.login || (!params.pass && params.agent && !params.agent.includes('MinerGate'))) { - sendReply("No login/password specified"); + for (const param of params) if (typeof param !== 'string') { + sendReply("Not correct params specified"); return; } - let difficulty = portData.difficulty; - let minerId = uuidV4(); - miner = new Miner(minerId, params.login, params.pass, ip, difficulty, pushMessage, 1, portData.portType, portData.port, params.agent); - if (!miner.valid_miner) { - console.log("Invalid miner, disconnecting due to: " + miner.error); - sendReply(miner.error); + + if (params.length >= 3) params = { + job_id: params[1], + raw_params: params + }; else { + sendReply("Not correct params specified"); return; } - process.send({type: 'newMiner', data: miner.port}); - activeMiners[minerId] = miner; - sendReply(null, { - id: minerId, - job: miner.getJob(), - status: 'OK' - }); - break; - case 'getjob': - if (!miner) { - sendReply('Unauthenticated'); + + // continue to normal login + + case 'submit': { // grin and default + if (!params) { + sendReplyFinal("No params specified"); return; } - miner.heartbeat(); - miner.sendNewJob(); - break; - case 'submit': + const minerId = params.id ? params.id : (socket.miner_id ? socket.miner_id : ""); + let miner = activeMiners.get(minerId); if (!miner) { - sendReply('Unauthenticated'); + sendReplyFinal("Unauthenticated"); return; } + //if (miner.debugMiner) console.log("SUBMIT"); miner.heartbeat(); + if (typeof (params.job_id) === 'number') params.job_id = params.job_id.toString(); // for grin miner let job = miner.validJobs.toarray().filter(function (job) { return job.id === params.job_id; })[0]; if (!job) { - sendReply('Invalid job id'); + sendReply("Invalid job id"); return; } - params.nonce = params.nonce.substr(0, 8).toLowerCase(); - if (!nonceCheck.test(params.nonce)) { + const blob_type_num = job.blob_type_num; + + if (method === 'mining.submit') { + if (global.coinFuncs.blobTypeEth(blob_type_num) || global.coinFuncs.blobTypeErg(blob_type_num)) { + params.nonce = params.raw_params[2]; + } else if (global.coinFuncs.blobTypeRvn(blob_type_num) && params.raw_params.length >= 5) { + params.nonce = params.raw_params[2].substr(2); + params.header_hash = params.raw_params[3].substr(2); + params.mixhash = params.raw_params[4].substr(2); + } else { + sendReply("Invalid job params"); + return; + } + } + + const nonce_sanity_check = function(blob_type_num, params) { + if (global.coinFuncs.blobTypeGrin(blob_type_num)) { + if (typeof params.nonce !== 'number') return false; + if (!(params.pow instanceof Array)) return false; + if (params.pow.length != global.coinFuncs.c29ProofSize(blob_type_num)) return false; + } else { + if (typeof params.nonce !== 'string') return false; + if (global.coinFuncs.nonceSize(blob_type_num) == 8) { + const isExtraNonceBT = global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num); + if (isExtraNonceBT) params.nonce = job.extraNonce + params.nonce; + if (!nonceCheck64.test(params.nonce)) return false; + if (global.coinFuncs.blobTypeRvn(blob_type_num)) { + if (!hashCheck32.test(params.mixhash)) return false; + if (!hashCheck32.test(params.header_hash)) return false; + } else if (!isExtraNonceBT) { + if (!hashCheck32.test(params.result)) return false; + } + } else { + if (!nonceCheck32.test(params.nonce)) return false; + if (!hashCheck32.test(params.result)) return false; + } + } + return true; + }; + if (!nonce_sanity_check(blob_type_num, params)) { console.warn(threadName + 'Malformed nonce: ' + JSON.stringify(params) + ' from ' + miner.logString); miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); + sendReply("Duplicate share"); + miner.storeInvalidShare(); return; } - if (!miner.proxy) { - if (job.submissions.indexOf(params.nonce) !== -1) { - console.warn(threadName + 'Duplicate share: ' + JSON.stringify(params) + ' from ' + miner.logString); + + let nonce_test; + + if (miner.proxy) { + if (!Number.isInteger(params.poolNonce) || !Number.isInteger(params.workerNonce)) { + console.warn(threadName + 'Malformed nonce: ' + JSON.stringify(params) + ' from ' + miner.logString); miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); + sendReply("Duplicate share"); + miner.storeInvalidShare(); return; } - job.submissions.push(params.nonce); + nonce_test = global.coinFuncs.blobTypeGrin(blob_type_num) ? + params.pow.join(':') + `_${params.poolNonce}_${params.workerNonce}` : + `${params.nonce}_${params.poolNonce}_${params.workerNonce}`; } else { - if (!Number.isInteger(params.poolNonce) || !Number.isInteger(params.workerNonce)) { - console.warn(threadName + 'Malformed nonce: ' + JSON.stringify(params) + ' from ' + miner.logString); - miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); + nonce_test = global.coinFuncs.blobTypeGrin(blob_type_num) ? params.pow.join(':') : params.nonce; + } + + if (nonce_test in job.submissions) { + console.warn(threadName + 'Duplicate miner share with ' + nonce_test + ' nonce from ' + miner.logString); + miner.checkBan(false); + sendReply("Duplicate share"); + miner.storeInvalidShare(); + return; + } + job.submissions[nonce_test] = 1; + + let blockTemplate; + job.rewarded_difficulty = job.difficulty; + + if (activeBlockTemplates[job.coin].idHash !== job.blockHash) { + blockTemplate = pastBlockTemplates[job.coin].toarray().filter(function (t) { + return t.idHash === job.blockHash; + })[0]; + let is_outdated = false; + if (blockTemplate && blockTemplate.timeoutTime) { + const late_time = Date.now() - blockTemplate.timeoutTime; + if (late_time > 0) { + const max_late_time = global.config.pool.targetTime * 1000; + if (late_time < max_late_time) { + let factor = (max_late_time - late_time) / max_late_time; + job.rewarded_difficulty = job.difficulty * Math.pow(factor, 6); //Math.floor(job.difficulty * Math.pow(factor, 6)); + //if (job.rewarded_difficulty === 0) job.rewarded_difficulty = 1; + } else { + is_outdated = true; + } + } + } + if (!blockTemplate || is_outdated) { + const err_str = blockTemplate ? "Block outdated" : "Block expired"; + const time_now = Date.now(); + if (!(miner.payout in lastMinerLogTime) || time_now - lastMinerLogTime[miner.payout] > 30*1000) { + console.warn(threadName + err_str + ', Height: ' + job.height + ' (diff ' + job.difficulty + ') from ' + miner.logString); + lastMinerLogTime[miner.payout] = time_now; + } + miner.sendSameCoinJob(); + sendReply(err_str); + miner.storeInvalidShare(); return; } - let nonce_test = `${params.nonce}_${params.poolNonce}_${params.workerNonce}`; - if (job.submissions.indexOf(nonce_test) !== -1) { - console.warn(threadName + 'Duplicate share: ' + JSON.stringify(params) + ' from ' + miner.logString); - miner.checkBan(false); - sendReply('Duplicate share'); - global.database.storeInvalidShare(miner.invalidShareProto); + } else { + blockTemplate = activeBlockTemplates[job.coin]; + // kill miner if it mines block template for disabled coin for more than some time + if (!lastCoinHashFactorMM[job.coin] && Date.now() - blockTemplate.timeCreated > 60*60*1000) { + sendReplyFinal("This algo was temporary disabled due to coin daemon issues. Consider using https://github.com/MoneroOcean/meta-miner to allow your miner auto algo switch in this case."); return; } - job.submissions.push(nonce_test); } - let blockTemplate = activeBlockTemplate.height === job.height ? activeBlockTemplate : pastBlockTemplates.toarray().filter(function (t) { - return t.height === job.height; - })[0]; + job.rewarded_difficulty2 = job.rewarded_difficulty * job.coinHashFactor; + //job.rewarded_difficulty = Math.floor(job.rewarded_difficulty); + //if (job.rewarded_difficulty === 0) job.rewarded_difficulty = 1; - if (!blockTemplate) { - console.warn(threadName + 'Block expired, Height: ' + job.height + ' from ' + miner.logString); - if (miner.incremented === false) { - miner.newDiff = miner.difficulty + 1; - miner.incremented = true; - } else { - miner.newDiff = miner.difficulty - 1; - miner.incremented = false; + processShare(miner, job, blockTemplate, params, function(shareAccepted) { + if (miner.removed_miner) return; + if (shareAccepted === null) { + sendReply('Throttled down share submission (please increase difficulty)'); + return; } - miner.sendNewJob(); - sendReply('Block expired'); - global.database.storeInvalidShare(miner.invalidShareProto); - return; - } - - let shareAccepted = processShare(miner, job, blockTemplate, params); - miner.checkBan(shareAccepted); - - if (global.config.pool.trustedMiners) { - if (shareAccepted) { - miner.trust.probability -= global.config.pool.trustChange; - if (miner.trust.probability < (global.config.pool.trustMin)) { - miner.trust.probability = global.config.pool.trustMin; + miner.checkBan(shareAccepted); + + if (global.config.pool.trustedMiners) { + if (shareAccepted) { + miner.trust.trust += job.rewarded_difficulty2; + miner.trust.check_height = 0; + } else { + debug(threadName + "Share trust broken by " + miner.logString); + miner.storeInvalidShare(); + miner.trust.trust = 0; } - miner.trust.penalty--; - miner.trust.threshold--; } - else { - console.log(threadName + "Share trust broken by " + miner.logString); - global.database.storeInvalidShare(miner.invalidShareProto); - miner.trust.probability = 256; - miner.trust.penalty = global.config.pool.trustPenalty; - miner.trust.threshold = global.config.pool.trustThreshold; - } - } - if (!shareAccepted) { - sendReply('Low difficulty share'); - return; - } + if (!shareAccepted) { + sendReply("Low difficulty share"); + return; + } - let now = Date.now() / 1000 || 0; - miner.shareTimeBuffer.enq(now - miner.lastShareTime); - miner.lastShareTime = now; + miner.lastShareTime = Date.now() / 1000 || 0; - sendReply(null, {status: 'OK'}); + if (miner.protocol === "grin") { + sendReply(null, "ok"); + } else if ( global.coinFuncs.blobTypeRvn(blob_type_num) || + global.coinFuncs.blobTypeEth(blob_type_num) || + global.coinFuncs.blobTypeErg(blob_type_num) + ) { + sendReply(null, true); + } else { + sendReply(null, { status: 'OK' }); + } + //if (miner.debugMiner) console.log("SUBMIT OK"); + }); break; - case 'keepalived': + } + + case 'keepalive': + case 'keepalived': { + if (!params) { + sendReplyFinal("No params specified"); + return; + } + const minerId = params.id ? params.id : (socket.miner_id ? socket.miner_id : ""); + let miner = activeMiners.get(minerId); if (!miner) { - sendReply('Unauthenticated'); + sendReplyFinal("Unauthenticated"); return; } - sendReply(null, { - status: 'KEEPALIVED' - }); + miner.heartbeat(); + sendReply(null, { status: 'KEEPALIVED' }); break; + } } } -if (cluster.isMaster) { - let numWorkers = require('os').cpus().length; - global.config.ports.forEach(function (portData) { - minerCount[portData.port] = 0; +if (global.config.general.allowStuckPoolKill && fs.existsSync("block_template_is_stuck")) { + console.error("Stuck block template was detected on previous run. Please fix monerod and remove block_template_is_stuck file after that. Exiting..."); + setTimeout(function() { process.exit(); }, 5*1000); + return; +} + +setInterval(function dump_vars() { + const fn = "dump" + (cluster.isMaster ? "" : "_" + process.env['WORKER_ID'].toString()); + fs.access(fn, fs.F_OK, function(err) { + if (!err) return; + console.log("DUMPING VARS TO " + fn + " FILE"); + let s = fs.createWriteStream(fn, {'flags': 'a'}); + + s.write("activeMiners:\n"); + for (var [minerId, miner] of activeMiners) s.write(minerId + ": " + JSON.stringify(miner, null, '\t') + "\n"); + + s.write("\n\n\npastBlockTemplates:\n"); + s.write(JSON.stringify(pastBlockTemplates, null, '\t') + "\n"); + + s.write("\n\n\nlastBlockHash:\n"); + s.write(JSON.stringify(lastBlockHash, null, '\t') + "\n"); + + s.write("\n\n\nlastCoinHashFactor:\n"); + s.write(JSON.stringify(lastCoinHashFactor, null, '\t') + "\n"); + + s.write("\n\n\nnewCoinHashFactor:\n"); + s.write(JSON.stringify(newCoinHashFactor, null, '\t') + "\n"); + + s.write("\n\n\nlastCoinHashFactorMM:\n"); + s.write(JSON.stringify(lastCoinHashFactorMM, null, '\t') + "\n"); + + s.write("\n\n\nactiveBlockTemplates:\n"); + s.write(JSON.stringify(activeBlockTemplates, null, '\t') + "\n"); + + s.write("\n\n\nproxyMiners:\n"); + s.write(JSON.stringify(proxyMiners, null, '\t') + "\n"); + + s.write("\n\n\nanchorBlockHeight: " + anchorBlockHeight + "\n"); + s.write("\n\n\nanchorBlockPrevHeight: " + anchorBlockPrevHeight + "\n"); + + s.write("\n\n\nwalletTrust:\n"); + s.write(JSON.stringify(walletTrust, null, '\t') + "\n"); + + s.write("\n\n\nwalletLastSeeTime:\n"); + s.write(JSON.stringify(walletLastSeeTime, null, '\t') + "\n"); + + s.write("\n\n\nwalletAcc:\n"); + s.write(JSON.stringify(walletAcc, null, '\t') + "\n"); + + s.write("\n\n\nwalletWorkerCount:\n"); + s.write(JSON.stringify(walletWorkerCount, null, '\t') + "\n"); + + s.write("\n\n\nis_walletAccFinalizer:\n"); + s.write(JSON.stringify(is_walletAccFinalizer, null, '\t') + "\n"); + + s.write("\n\n\nbannedTmpIPs:\n"); + s.write(JSON.stringify(bannedTmpIPs, null, '\t') + "\n"); + + s.write("\n\n\nbannedTmpWallets:\n"); + s.write(JSON.stringify(bannedTmpWallets, null, '\t') + "\n"); + + s.write("\n\n\nbannedBigTmpWallets:\n"); + s.write(JSON.stringify(bannedBigTmpWallets, null, '\t') + "\n"); + + s.end(); + }); +}, 60*1000); + +let master_cluster_worker_id_map = {}; + +function getUniqueWorkerID(cb) { + if (!global.config.eth_pool_support) return cb(0, 1); + global.mysql.query("SELECT id FROM pool_workers WHERE pool_id = ? AND worker_id = ?", [global.config.pool_id, process.env['WORKER_ID']]).then(function (rows) { + if (rows.length === 0) { + global.mysql.query("INSERT INTO pool_workers (pool_id, worker_id) VALUES (?, ?) ON DUPLICATE KEY UPDATE id=id", [global.config.pool_id, process.env['WORKER_ID']]).then(function() { + return getUniqueWorkerID(cb); + }).catch(function(err) { + console.error("Can't register unique pool worker for " + global.config.pool_id + " pool_id and " + process.env['WORKER_ID'] + " worker_id"); + process.exit(1); + }); + } else if (rows.length !== 1) { + console.error("Can't get unique pool worker for " + global.config.pool_id + " pool_id and " + process.env['WORKER_ID'] + " worker_id"); + process.exit(1); + } else global.mysql.query("SELECT MAX(id) as maxId FROM pool_workers").then(function (rows_max) { + if (rows_max.length !== 1) { + console.error("Can't get max id from pool_workers table"); + process.exit(1); + } + if (global.config.max_pool_worker_num && rows_max[0].maxId > global.config.max_pool_worker_num) { + console.error("Prease recreate pool_workers table"); + process.exit(1); + } + return cb(rows[0].id - 1, (global.config.max_pool_worker_num ? global.config.max_pool_worker_num : rows_max[0].maxId) - 1); + }); }); +} + +if (cluster.isMaster) { + const numWorkers = global.config.worker_num ? global.config.worker_num : require('os').cpus().length; + for (let i = 1; i <= numWorkers; ++ i) { + minerCount[i] = []; + global.config.ports.forEach(function (portData) { + minerCount[i][portData.port] = 0; + }); + } registerPool(); + setInterval(function () { - global.mysql.query("UPDATE pools SET last_checkin = ?, active = ? WHERE id = ?", [global.support.formatDate(Date.now()), true, global.config.pool_id]); - global.mysql.query("UPDATE pools SET blockIDTime = now(), blockID = ? where id = ?", [activeBlockTemplate.height, global.config.pool_id]); + if ("" in activeBlockTemplates) { + global.mysql.query("UPDATE pools SET last_checkin = ?, active = ?, blockIDTime = now(), blockID = ?, port = ? WHERE id = ?", [global.support.formatDate(Date.now()), true, activeBlockTemplates[""].height, activeBlockTemplates[""].port, global.config.pool_id]).catch(function (error) { + console.error("SQL query failed: " + error); + }); + } else { + global.mysql.query("UPDATE pools SET last_checkin = ?, active = ? WHERE id = ?", [global.support.formatDate(Date.now()), true, global.config.pool_id]).catch(function (error) { + console.error("SQL query failed: " + error); + }); + } global.config.ports.forEach(function (portData) { - global.mysql.query("UPDATE ports SET lastSeen = now(), miners = ? WHERE pool_id = ? AND network_port = ?", [minerCount[portData.port], global.config.pool_id, portData.port]); + let miner_count = 0; + for (let i = 1; i <= numWorkers; ++ i) miner_count += minerCount[i][portData.port]; + global.mysql.query("UPDATE ports SET lastSeen = now(), miners = ? WHERE pool_id = ? AND network_port = ?", [miner_count, global.config.pool_id, portData.port]).catch(function (error) { + console.error("SQL query failed: " + error); + }); + }); + }, 30*1000); + + + setInterval(function () { + if (!("" in activeBlockTemplates)) return; + + global.mysql.query("SELECT blockID, port FROM pools WHERE last_checkin > date_sub(now(), interval 30 minute)").then(function (rows) { + let top_height = 0; + const port = activeBlockTemplates[""].port; + const height = activeBlockTemplates[""].height; + rows.forEach(function (row) { + if (row.port != port) return; + if (row.blockID > top_height) top_height = row.blockID; + }); + if (top_height) { + if (height < top_height - 3) { + console.error("!!! Current block height " + height + " is stuck compared to top height (" + top_height + ") amongst other leaf nodes for " + port + " port"); + if (!(port in lastBlockFixTime)) lastBlockFixTime[port] = Date.now(); + + if (Date.now() - lastBlockFixTime[port] > 20*60*1000) { + if (!(port in lastBlockFixCount)) lastBlockFixCount[port] = 1; + else ++ lastBlockFixCount[port]; + + if (lastBlockFixCount[port] > 5 && global.config.general.allowStuckPoolKill && port == global.config.daemon.port) { + global.support.sendEmail(global.config.general.adminEmail, + "Pool server " + global.config.hostname + " will be terminated", + "The pool server: " + global.config.hostname + " with IP: " + global.config.bind_ip + " will be terminated due to main chain block template stuck" + ); + console.error("Block height was not updated for a long time for main port. Check your monerod. Exiting..."); + fs.closeSync(fs.openSync("block_template_is_stuck", 'w')); + setTimeout(function() { process.exit(); }, 30*1000); // need time for admin email sending + return; + } + + global.coinFuncs.fixDaemonIssue(height, top_height, port); + lastBlockFixTime[port] = Date.now(); + } + } else { + if (height >= top_height + 3) { + console.warn("Current block height " + height + " is somehow greater than top height (" + top_height + ") amongst other leaf nodes for " + port + " port"); + } + lastBlockFixTime[port] = Date.now(); + lastBlockFixCount[port] = 0; + } + } else { + console.error("Can't get top height amongst all leaf nodes for " + port + " port"); + lastBlockFixTime[port] = Date.now(); + lastBlockFixCount[port] = 0; + } + }).catch(function (error) { + console.error("SQL query failed: " + error); }); - }, 10000); + }, 60*1000); + console.log('Master cluster setting up ' + numWorkers + ' workers...'); for (let i = 0; i < numWorkers; i++) { - let worker = cluster.fork(); + let worker = cluster.fork({ WORKER_ID: master_cluster_worker_id_map[i + 1] = i + 1 }); worker.on('message', messageHandler); - workerList.push(worker); } cluster.on('online', function (worker) { @@ -848,23 +2471,212 @@ if (cluster.isMaster) { }); cluster.on('exit', function (worker, code, signal) { - console.log('Worker ' + worker.process.pid + ' died with code: ' + code + ', and signal: ' + signal); + console.error('Worker ' + worker.process.pid + ' died with code: ' + code + ', and signal: ' + signal); console.log('Starting a new worker'); - worker = cluster.fork(); + const prev_worker_id = master_cluster_worker_id_map[worker.id]; + delete master_cluster_worker_id_map[worker.id]; + worker = cluster.fork({ WORKER_ID: prev_worker_id }); + master_cluster_worker_id_map[worker.id] = prev_worker_id; worker.on('message', messageHandler); - workerList.push(worker); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Started new worker " + prev_worker_id, + "Hello,\r\nMaster thread of " + global.config.hostname + " starts new worker with id " + prev_worker_id); }); - templateUpdate(); - setInterval(templateUpdate, 300, true); + + + newCoinHashFactor[""] = lastCoinHashFactor[""] = lastCoinHashFactorMM[""] = 1; + templateUpdate(""); + setTimeout(templateUpdate, DAEMON_POLL_MS, "", true); + + if (global.config.daemon.enableAlgoSwitching) { + if (global.config.daemon.enableAlgoSwitching) COINS.forEach(function(coin) { + newCoinHashFactor[coin] = lastCoinHashFactor[coin] = lastCoinHashFactorMM[coin] = 0; + setInterval(updateCoinHashFactor, 5*1000, coin); + templateUpdate(coin); + setTimeout(templateUpdate, DAEMON_POLL_MS, coin, true); + }); + } else { + console.warn("global.config.daemon.enableAlgoSwitching is not enabled"); + } + global.support.sendEmail(global.config.general.adminEmail, "Pool server " + global.config.hostname + " online", "The pool server: " + global.config.hostname + " with IP: " + global.config.bind_ip + " is online"); -} else { - setInterval(checkAliveMiners, 30000); + + let block_notify_server = net.createServer(function (socket) { + let timer = setTimeout(function() { + console.error(threadName + "Timeout waiting for block notify input"); + socket.destroy(); + }, 3*1000); + let buff = ""; + socket.on('data', function (buff1) { + buff += buff1; + }); + socket.on('end', function () { + clearTimeout(timer); + timer = null; + const port = parseInt(buff.toString()); + const coin = global.coinFuncs.PORT2COIN(port); + if (typeof(coin) === 'undefined') { + console.error(threadName + "Block notify for unknown coin with " + port + " port"); + } else { + //console.log(threadName + "Block notify for coin " + coin + " with " + port + " port"); + templateUpdate(coin, false); + } + }); + socket.on('error', function() { + console.error(threadName + "Socket error on block notify port"); + socket.destroy(); + }); + }); + + block_notify_server.listen(BLOCK_NOTIFY_PORT, "127.0.0.1", function() { + console.log(threadName + "Block notify server on " + BLOCK_NOTIFY_PORT + " port started"); + }); + +} else getUniqueWorkerID(function(id, maxId) { + uniqueWorkerId = id; + uniqueWorkerIdBits = 0; + while (maxId) { maxId >>= 1; ++ uniqueWorkerIdBits; } + freeEthExtranonces = [...Array(1 << (16 - uniqueWorkerIdBits)).keys()]; + console.log(threadName + "Starting pool worker with " + uniqueWorkerId + " unique id and " + uniqueWorkerIdBits + " reserved bits"); + + newCoinHashFactor[""] = lastCoinHashFactor[""] = lastCoinHashFactorMM[""] = 1; + templateUpdate(""); + if (global.config.daemon.enableAlgoSwitching) COINS.forEach(function(coin) { + newCoinHashFactor[coin] = lastCoinHashFactor[coin] = lastCoinHashFactorMM[coin] = 0; + templateUpdate(coin); + }); + anchorBlockUpdate(); + setInterval(anchorBlockUpdate, 3*1000); + setInterval(checkAliveMiners, 60*1000); setInterval(retargetMiners, global.config.pool.retargetTime * 1000); - templateUpdate(); setInterval(function () { - bannedIPs = []; - templateUpdate(); - }, 60000); + bannedTmpIPs = {}; + bannedTmpWallets = {}; + }, 10*60*1000); + + setInterval(function () { + bannedBigTmpWallets = {}; + }, 60*60*1000); + + function add_bans(is_show) { + global.mysql.query("SELECT mining_address, reason FROM bans").then(function (rows) { + bannedAddresses = {}; + rows.forEach(function (row) { + bannedAddresses[row.mining_address] = row.reason; + if (is_show) console.log("Added blocked address " + row.mining_address + ": " + row.reason); + }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); + global.mysql.query("SELECT mining_address, message FROM notifications").then(function (rows) { + notifyAddresses = {}; + rows.forEach(function (row) { + notifyAddresses[row.mining_address] = row.message; + if (is_show) console.log("Added notify address " + row.mining_address + ": " + row.message); + }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); + } + + add_bans(true); + setInterval(add_bans, 10*60*1000); + + // load merged wallet trust from files + let numWorkers = require('os').cpus().length; + for (let i = 1; i <= numWorkers; ++ i) { + let fn = "wallet_trust_" + i.toString(); + let rs = fs.createReadStream(fn); + rs.on('error', function() { console.error("Can't open " + fn + " file"); }); + let lineReader = require('readline').createInterface({ input: rs }); + lineReader.on('error', function() { console.error("Can't read lines from " + fn + " file"); }); + lineReader.on('line', function (line) { + let parts = line.split(/\t/); + if (parts.length != 3) { + console.error("Error line " + line + " ignored from " + fn + " file"); + return; + } + let wallet = parts[0]; + let trust = parseInt(parts[1], 10); + let time = parseInt(parts[2], 10); + if (Date.now() - time < 24*60*60*1000 && (!(wallet in walletTrust) || trust < walletTrust[wallet])) { + debug("Adding " + trust.toString() + " trust for " + wallet + " wallet"); + walletTrust[wallet] = trust; + walletLastSeeTime[wallet] = time; + } + }); + } + + // dump wallet trust and miner agents to file + setInterval(function () { + let str = ""; + for (let wallet in walletTrust) { + let time = walletLastSeeTime[wallet]; + if (Date.now() - time < 24*60*60*1000) { + str += wallet + "\t" + walletTrust[wallet].toString() + "\t" + time.toString() + "\n"; + } else { + delete walletTrust[wallet]; + delete walletLastSeeTime[wallet]; + } + } + const fn = "wallet_trust_" + process.env['WORKER_ID'].toString(); + fs.writeFile(fn, str, function(err) { if (err) console.error("Error saving " + fn + " file"); }); + + if (process.env['WORKER_ID'] == 1) { + let str2 = ""; + for (let agent in minerAgents) { str2 += agent + "\n"; } + const fn2 = "miner_agents"; + fs.writeFile(fn2, str2, function(err) { if (err) console.error("Error saving " + fn2 + " file"); }); + } + + //cacheTargetHex = {}; + + }, 10*60*1000); + + // get extra wallets to check + setInterval(function () { + const extra_wallet_verify_fn = "extra_wallet_verify.txt"; + extra_wallet_verify = {}; + fs.access(extra_wallet_verify_fn, fs.F_OK, function(err) { + if (err) return; + let rs = fs.createReadStream(extra_wallet_verify_fn); + rs.on('error', function() { console.error("Can't open " + extra_wallet_verify_fn + " file"); }); + let lineReader = require('readline').createInterface({ input: rs }); + lineReader.on('line', function (line) { + console.log(threadName + "[EXTRA CHECK] added: '" + line + "'"); + extra_wallet_verify[line] = 1; + }); + const fn = "extra_verify_wallet_hashes_" + process.env['WORKER_ID'].toString(); + fs.writeFile(fn, extra_verify_wallet_hashes.join("\n"), function(err) { if (err) console.error("Error saving " + fn + " file"); }); + extra_verify_wallet_hashes = []; + }); + const wallet_debug_fn = "wallet_debug.txt"; + wallet_debug = {}; + fs.access(wallet_debug_fn, fs.F_OK, function(err) { + if (err) return; + let rs = fs.createReadStream(wallet_debug_fn); + rs.on('error', function() { console.error("Can't open " + wallet_debug_fn + " file"); }); + let lineReader = require('readline').createInterface({ input: rs }); + lineReader.on('line', function (line) { + console.log(threadName + "[WALLET DEBUG] added: '" + line + "'"); + wallet_debug[line] = 1; + }); + }); + const ip_whitelist_fn = "ip_whitelist.txt"; + ip_whitelist = {}; + fs.access(ip_whitelist_fn, fs.F_OK, function(err) { + if (err) return; + let rs = fs.createReadStream(ip_whitelist_fn); + rs.on('error', function() { console.error("Can't open " + ip_whitelist_fn + " file"); }); + let lineReader = require('readline').createInterface({ input: rs }); + lineReader.on('line', function (line) { + console.log(threadName + "[IP WHITELIST]: '" + line + "'"); + ip_whitelist[line] = 1; + }); + }); + }, 5*60*1000); + + //let lastGarbageFromIpTime = {}; + async.each(global.config.ports, function (portData) { if (global.config[portData.portType].enable !== true) { return; @@ -873,29 +2685,42 @@ if (cluster.isMaster) { if (!jsonData.id) { console.warn('Miner RPC request missing RPC id'); return; - } - else if (!jsonData.method) { + } else if (!jsonData.method) { console.warn('Miner RPC request missing RPC method'); return; } - else if (!jsonData.params) { - console.warn('Miner RPC request missing RPC params'); - return; - } let sendReply = function (error, result) { - if (!socket.writable) { - return; - } - let sendData = JSON.stringify({ - id: jsonData.id, - jsonrpc: "2.0", - error: error ? {code: -1, message: error} : null, - result: result - }) + "\n"; - socket.write(sendData); + if (!socket.writable) return; + let reply = { + jsonrpc: "2.0", + id: jsonData.id, + error: error ? {code: -1, message: error} : null, + result: result + }; + if (jsonData.id === "Stratum") reply.method = jsonData.method; + debug("[MINER] REPLY TO MINER: " + JSON.stringify(reply)); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] reply " + JSON.stringify(reply)); + socket.write(JSON.stringify(reply) + "\n"); }; - handleMinerData(jsonData.method, jsonData.params, socket.remoteAddress, portData, sendReply, pushMessage); + let sendReplyFinal = function (error, timeout) { + setTimeout(function() { + if (!socket.writable) return; + let reply = { + jsonrpc: "2.0", + id: jsonData.id, + error: {code: -1, message: error}, + result: null + }; + if (jsonData.id === "Stratum") reply.method = jsonData.method; + debug("[MINER] FINAL REPLY TO MINER: " + JSON.stringify(reply)); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] final reply " + JSON.stringify(reply)); + socket.end(JSON.stringify(reply) + "\n"); + }, (timeout ? timeout : 9) * 1000); + }; + debug("[MINER] GOT FROM MINER: " + JSON.stringify(jsonData)); + handleMinerData(socket, jsonData.id, jsonData.method, jsonData.params, socket.remoteAddress, portData, sendReply, sendReplyFinal, pushMessage); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] recieved " + JSON.stringify(jsonData)); }; function socketConn(socket) { @@ -904,21 +2729,17 @@ if (cluster.isMaster) { let dataBuffer = ''; - let pushMessage = function (method, params) { - if (!socket.writable) { - return; - } - let sendData = JSON.stringify({ - jsonrpc: "2.0", - method: method, - params: params - }) + "\n"; - socket.write(sendData); + let pushMessage = function (body) { + if (!socket.writable) return; + body.jsonrpc = "2.0"; + debug("[MINER] PUSH TO MINER: " + JSON.stringify(body)); + if (socket.debugMiner) console.log(threadName + " [WALLET DEBUG] push " + JSON.stringify(body)); + socket.write(JSON.stringify(body) + "\n"); }; socket.on('data', function (d) { dataBuffer += d; - if (Buffer.byteLength(dataBuffer, 'utf8') > 10240) { //10KB + if (Buffer.byteLength(dataBuffer, 'utf8') > 102400) { //100KB dataBuffer = null; console.warn(threadName + 'Excessive packet size from: ' + socket.remoteAddress); socket.destroy(); @@ -937,18 +2758,22 @@ if (cluster.isMaster) { jsonData = JSON.parse(message); } catch (e) { - if (message.indexOf('GET /') === 0) { - if (message.indexOf('HTTP/1.1') !== -1) { - socket.end('HTTP/1.1' + httpResponse); - break; - } - else if (message.indexOf('HTTP/1.0') !== -1) { - socket.end('HTTP/1.0' + httpResponse); - break; - } - } - - console.warn(threadName + "Malformed message from " + socket.remoteAddress + " Message: " + message); + //if (message.indexOf('GET /') === 0) { + // if (message.indexOf('HTTP/1.1') !== -1) { + // socket.end('HTTP/1.1' + httpResponse); + // break; + // } + // else if (message.indexOf('HTTP/1.0') !== -1) { + // socket.end('HTTP/1.0' + httpResponse); + // break; + // } + //} + + //let time_now = Date.now(); + //if (!(socket.remoteAddress in lastGarbageFromIpTime) || time_now - lastGarbageFromIpTime[socket.remoteAddress] > 60*1000) { + // console.warn(threadName + "Malformed message from " + socket.remoteAddress + " Message: " + JSON.stringify(message)); + // lastGarbageFromIpTime[socket.remoteAddress] = time_now; + //} socket.destroy(); break; @@ -958,34 +2783,41 @@ if (cluster.isMaster) { dataBuffer = incomplete; } }).on('error', function (err) { - if (err.code !== 'ECONNRESET') { - console.warn(threadName + "Socket Error from " + socket.remoteAddress + " Error: " + err); - } + //debug(threadName + "Socket Error " + err.code + " from " + socket.remoteAddress + " Error: " + err); }).on('close', function () { - pushMessage = function () { - }; + pushMessage = function () {}; + if (socket.miner_id) activeMiners.get(socket.miner_id); + if ("eth_extranonce_id" in socket) freeEthExtranonces.push(socket.eth_extranonce_id); }); } if ('ssl' in portData && portData.ssl === true) { - tls.createServer({ + let server = tls.createServer({ key: fs.readFileSync('cert.key'), cert: fs.readFileSync('cert.pem') - }, socketConn).listen(portData.port, global.config.bind_ip, function (error) { + }, socketConn); + server.listen(portData.port, global.config.bind_ip, function (error) { if (error) { console.error(threadName + "Unable to start server on: " + portData.port + " Message: " + error); return; } console.log(threadName + "Started server on port: " + portData.port); }); + server.on('error', function (error) { + console.error("Can't bind server to " + portData.port + " SSL port!"); + }); } else { - net.createServer(socketConn).listen(portData.port, global.config.bind_ip, function (error) { + let server = net.createServer(socketConn); + server.listen(portData.port, global.config.bind_ip, function (error) { if (error) { console.error(threadName + "Unable to start server on: " + portData.port + " Message: " + error); return; } console.log(threadName + "Started server on port: " + portData.port); }); + server.on('error', function (error) { + console.error("Can't bind server to " + portData.port + " port!"); + }); } }); -} +}); diff --git a/lib/pool_stats.js b/lib/pool_stats.js new file mode 100644 index 00000000..d6d07027 --- /dev/null +++ b/lib/pool_stats.js @@ -0,0 +1,486 @@ +"use strict"; +const debug = require("debug")("pool_stats"); +const async = require("async"); + +const threadName = "Worker Server "; +const max_blocks = 1000; +const max_altblocks = 10000; + +let lastBlockCheckIsFailed = {}; + +let price_btc = 0; +let price_usd = 0; +let price_eur = 0; +let min_block_rewards = {}; +let blockList = []; +let altblockList = []; +let altblockFound = {}; +let altblockFoundDone = 0; + +function get_cmc_price(symbol, callback) { + const slug = global.config.coin.name.toLowerCase(); + global.support.https_get("https://pro-api.coinmarketcap.com/v1/cryptocurrency/quotes/latest?slug=" + slug + "&convert=" + symbol + "&CMC_PRO_API_KEY=" + global.config.general.cmcKey, function (res) { + if (res instanceof Object && "data" in res && "quote" in res.data[Object.keys(res.data)[0]] && symbol in res.data[Object.keys(res.data)[0]].quote) { + return callback(parseFloat(res.data[Object.keys(res.data)[0]].quote[symbol].price)); + } else { + console.error("Can't get price data from: " + JSON.stringify(res)); + return callback(0); + } + }); +} + +function get_cmc(callback) { + get_cmc_price("USD", function(usd) { + get_cmc_price("EUR", function(eur) { + get_cmc_price("BTC", function(btc) { + price_btc = btc ? btc : price_btc; + price_usd = usd ? usd : price_usd; + price_eur = eur ? eur : price_eur; + return callback({ btc: price_btc, usd: price_usd, eur: price_eur }); + }); + }); + }); +} + +function updatePoolStats(poolType) { + //console.log("Cleaned " + global.database.env.mdb_reader_check() + " stale readers"); + let cache; + if (typeof(poolType) !== 'undefined') { + cache = global.database.getCache(poolType + "_stats"); + let cache2 = global.database.getCache(poolType + "_stats2"); + cache.totalHashes = cache2.totalHashes; + cache.roundHashes = cache2.roundHashes; + } else { + console.log("Running pool stats"); + cache = global.database.getCache("global_stats"); + let cache2 = global.database.getCache("global_stats2"); + cache.totalHashes = cache2.totalHashes; + cache.roundHashes = cache2.roundHashes; + } + + let port_hash = global.database.getCache('port_hash'); + if (blockList.length > max_blocks) { + const newBlocks = global.database.getBlockList(poolType, 0, max_blocks); + let new_block_count = 0; + let prev_block_index = 0; + for (let block of newBlocks) { + if (block.hash == blockList[prev_block_index].hash) ++ prev_block_index; + else ++ new_block_count; + } + blockList = newBlocks.concat(blockList.slice(max_blocks - new_block_count)); + } else { + blockList = global.database.getBlockList(poolType); + } + if (altblockList.length > max_altblocks) { + const newBlocks = global.database.getAltBlockList(poolType, null, 0, max_altblocks); + let new_block_count = 0; + let prev_block_index = 0; + for (let block of newBlocks) { + if (block.hash == altblockList[prev_block_index].hash) ++ prev_block_index; + else ++ new_block_count; + } + altblockList = newBlocks.concat(altblockList.slice(max_altblocks - new_block_count)); + } else { + altblockList = global.database.getAltBlockList(poolType); + } + let min_block_rewards2 = global.database.getCache('min_block_rewards'); + if (min_block_rewards2) min_block_rewards = min_block_rewards2; + if (!(global.config.daemon.port in min_block_rewards)) min_block_rewards[global.config.daemon.port] = 0; + + async.series([ + function (callback) { + //debug(threadName + "Checking Influx for last 5min avg for pool stats (hashRate)"); + return callback(null, cache.hash || 0); + }, + function (callback) { + //debug(threadName + "Checking Influx for last 5min avg for miner count for pool stats (miners)"); + return callback(null, cache.minerCount || 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for totalHashes"); + return callback(null, cache.totalHashes || 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB for lastBlockFoundTime for pool stats"); + let max_time = 0; + if (blockList.length !== 0) { + max_time = Math.floor(blockList[0].ts / 1000); + } + if (altblockList.length !== 0) { + max_time = Math.max(max_time, Math.floor(altblockList[0].ts / 1000)); + } + return callback(null, max_time); + }, + function (callback) { + //debug(threadName + "Checking LMDB for lastBlockFound height for pool stats"); + if (blockList.length === 0) { + return callback(null, 0); + } + return callback(null, blockList[0].height); + }, + function (callback) { + //debug(threadName + "Checking LMDB for totalBlocksFound for pool stats"); + return callback(null, blockList.length); + }, + function (callback) { + //debug(threadName + "Checking MySQL for total miners paid"); + if (typeof(poolType) !== 'undefined') { + global.mysql.query("SELECT payment_address, payment_id FROM payments WHERE pool_type = ? group by payment_address, payment_id", [poolType]).then(function (rows) { + return callback(null, rows.length); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(null, 0); + }); + } else { + global.mysql.query("SELECT payment_address, payment_id FROM payments group by payment_address, payment_id").then(function (rows) { + return callback(null, rows.length); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(null, 0); + }); + } + }, + function (callback) { + //debug(threadName + "Checking MySQL for total transactions count"); + if (typeof(poolType) !== 'undefined') { + global.mysql.query("SELECT distinct(transaction_id) from payments WHERE pool_type = ?", [poolType]).then(function (rows) { + return callback(null, rows.length); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(null, 0); + }); + } else { + global.mysql.query("SELECT count(id) as txn_count FROM transactions").then(function (rows) { + if (typeof(rows[0]) !== 'undefined') { + return callback(null, rows[0].txn_count); + } else { + return callback(null, 0); + } + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(null, 0); + }); + } + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for roundHashes"); + return callback(null, cache.roundHashes || 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB for altblock count for pool stats"); + return callback(null, altblockList.length); + }, + function (callback) { + //debug(threadName + "Checking LMDB for altBlocksFound array for each specific port"); + for (let i in altblockList) { + if (i >= altblockList.length - altblockFoundDone) break; + let block = altblockList[i]; + if (altblockFound.hasOwnProperty(block.port)) ++ altblockFound[block.port]; + else altblockFound[block.port] = 1; + } + altblockFoundDone = altblockList.length; + return callback(null, altblockFound); + }, + function (callback) { + //debug(threadName + "Checking MySQL for activePort value"); + return callback(null, global.config.daemon.port); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for active_ports value"); + let active_ports = global.database.getCache('active_ports'); + return callback(null, active_ports ? active_ports : []); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for xmr_profit value"); + let xmr_profit = global.database.getCache('xmr_profit'); + return callback(null, xmr_profit ? xmr_profit.value : 0); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for coin_profit value"); + let coin_xmr_profit = global.database.getCache('coin_xmr_profit'); + return callback(null, coin_xmr_profit ? coin_xmr_profit : {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for xmr_profit_comment value"); + let coin_comment = global.database.getCache('coin_comment'); + return callback(null, coin_comment ? coin_comment : {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for min_block_rewards value to set minBlockRewards"); + return callback(null, min_block_rewards); + }, + function (callback) { + let pending = 0; + for (let i in blockList) { + if (i > max_blocks) break; + const block = blockList[i]; + if (block.valid === true && block.unlocked === false) pending += global.support.coinToDecimal(block.value); + } + for (let i in altblockList) { + if (i > max_altblocks) break; + const altblock = altblockList[i]; + if (altblock.valid === true && altblock.unlocked === false) pending += altblock.port in min_block_rewards ? min_block_rewards[altblock.port] : 0; + } + return callback(null, pending); + }, + function (callback) { + if (typeof(poolType) === 'undefined' && price_btc == 0 && price_usd == 0 && price_eur == 0) { + return get_cmc(function(prices) { return callback(null, prices); }); + } else return callback(null, { btc: price_btc, usd: price_usd, eur: price_eur }); + }, + function (callback) { + let currentEfforts = {}; + for (let port of global.coinFuncs.getPORTS()) { + const value = global.database.getCache(port != global.config.daemon.port ? "global_stats2_" + port : "global_stats2"); + if (value !== false) currentEfforts[port] = value.roundHashes; + } + return callback(null, currentEfforts); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for pplns_port_shares value"); + let pplns_port_shares = global.database.getCache('pplns_port_shares'); + return callback(null, pplns_port_shares ? pplns_port_shares : {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for pplns_window_time value"); + let pplns_window_time = global.database.getCache('pplns_window_time'); + return callback(null, pplns_window_time ? pplns_window_time : 0); + }, + function (callback) { + //debug(threadName + "Checking Influx for last 5min avg for pool stats (hashRate) per port"); + return callback(null, port_hash || {}); + }, + function (callback) { + //debug(threadName + "Checking LMDB cache for portMinerCount"); + return callback(null, global.database.getCache('portMinerCount') || {}); + }, + function (callback) { + let portCoinAlgo = {}; + for (let port of global.coinFuncs.getPORTS()) portCoinAlgo[port] = global.coinFuncs.algoShortTypeStr(port, 0); + return callback(null, portCoinAlgo); + }, + ], function (err, result) { + global.database.setCache('pool_stats_' + (typeof(poolType) === 'undefined' ? 'global' : poolType), { + hashRate: result[0], + miners: result[1], + totalHashes: result[2], + lastBlockFoundTime: result[3] || 0, + lastBlockFound: result[4] || 0, + totalBlocksFound: result[5] || 0, + totalMinersPaid: result[6] || 0, + totalPayments: result[7] || 0, + roundHashes: result[8] || 0, + totalAltBlocksFound: result[9] || 0, + altBlocksFound: result[10] || {}, + activePort: result[11] || 0, + activePorts: result[12] || [], + activePortProfit: result[13] || 0, + coinProfit: result[14] || {}, + coinComment: result[15] || {}, + minBlockRewards: result[16] || {}, + pending: result[17] || 0, + price: result[18] || {}, + currentEfforts: result[19] || {}, + pplnsPortShares: result[20] || {}, + pplnsWindowTime: result[21] || 0, + portHash: result[22] || {}, + portMinerCount: result[23] || {}, + portCoinAlgo: result[24] || {}, + }); + setTimeout(updatePoolStats, 60*1000, poolType); + }); +} + +function updatePoolPorts(poolServers) { + //debug(threadName + "Updating pool ports"); + let local_cache = {global: []}; + let portCount = 0; + global.mysql.query("select * from ports where hidden = 0 and pool_id < 1000 and lastSeen >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { + rows.forEach(function (row) { + ++ portCount; + if (!local_cache.hasOwnProperty(row.port_type)) { + local_cache[row.port_type] = []; + } + local_cache[row.port_type].push({ + host: poolServers[row.pool_id], + port: row.network_port, + difficulty: row.starting_diff, + description: row.description, + miners: row.miners + }); + if (portCount === rows.length) { + let local_counts = {}; + let port_diff = {}; + let port_miners = {}; + let pool_type_count = 0; + let localPortInfo = {}; + for (let pool_type in local_cache) { // jshint ignore:line + ++ pool_type_count; + local_cache[pool_type].forEach(function (portData) { // jshint ignore:line + if (!local_counts.hasOwnProperty(portData.port)) { + local_counts[portData.port] = 0; + } + if (!port_diff.hasOwnProperty(portData.port)) { + port_diff[portData.port] = portData.difficulty; + } + if (!port_miners.hasOwnProperty(portData.port)) { + port_miners[portData.port] = 0; + } + if (port_diff[portData.port] === portData.difficulty) { + ++ local_counts[portData.port]; + port_miners[portData.port] += portData.miners; + } + localPortInfo[portData.port] = portData.description; + if (local_counts[portData.port] === Object.keys(poolServers).length) { + local_cache.global.push({ + host: { + blockID: typeof(local_cache[pool_type][0].host) === 'undefined' ? 0 : local_cache[pool_type][0].host.blockID, + blockIDTime: typeof(local_cache[pool_type][0].host) === 'undefined' ? 0 : local_cache[pool_type][0].host.blockIDTime, + hostname: global.config.pool.geoDNS, + }, + port: portData.port, + pool_type: pool_type, + difficulty: portData.difficulty, + miners: port_miners[portData.port], + description: localPortInfo[portData.port] + }); + } + }); + if (pool_type_count === Object.keys(local_cache).length) { + //debug(threadName + "Sending the following to the workers: " + JSON.stringify(local_cache)); + global.database.setCache('poolPorts', local_cache); + setTimeout(updatePoolInformation, 30*1000); + } + } + } + }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); +} + +function updatePoolInformation() { + let local_cache = {}; + //debug(threadName + "Updating pool information"); + global.mysql.query("select * from pools where id < 1000 and last_checkin >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { + rows.forEach(function (row) { + local_cache[row.id] = { + ip: row.ip, + blockID: row.blockID, + blockIDTime: global.support.formatDateFromSQL(row.blockIDTime), + hostname: row.hostname + }; + if (Object.keys(local_cache).length === rows.length) { + global.database.setCache('poolServers', local_cache); + updatePoolPorts(local_cache); + } + }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); +} + +let network_info = {}; + +function updateBlockHeader() { + const ports = global.config.daemon.enableAlgoSwitching ? global.coinFuncs.getPORTS() : [ global.config.daemon.port ]; + async.eachSeries(ports, function(port, next) { + global.coinFuncs.getPortLastBlockHeader(port, function(err, body){ + if (err) return next(); + network_info[port] = { + difficulty: parseInt(body.difficulty), + hash: body.hash ? body.hash : body.hashrate, + height: body.height, + value: body.reward, + ts: parseInt(body.timestamp), + }; + if (port == global.config.daemon.port) { + global.support.rpcPortDaemon(port, 'get_info', [], function (rpcResult) { + network_info.difficulty = rpcResult.result ? rpcResult.result.difficulty : body.difficulty; + network_info.hash = body.hash; + network_info.main_height = body.height; + network_info.height = body.height; + network_info.value = body.reward; + network_info.ts = body.timestamp; + return next(); + }); + } else { + return next(); + } + }, true); + }, function(err, result) { + global.database.setCache('networkBlockInfo', network_info); + setTimeout(updateBlockHeader, 30*1000); + }); +} + +function bad_header_start(port) { + console.error("Issue in getting block header for " + port + " port. Skipping node monitor"); + if (port in lastBlockCheckIsFailed) { + if (++ lastBlockCheckIsFailed[port] >= 5) global.support.sendEmail( + global.config.general.adminEmail, + 'Failed to query daemon for ' + port + ' port for last block header', + `The worker failed to return last block header for ` + port + ` port. Please verify if the daemon is running properly.` + ); + } else { + lastBlockCheckIsFailed[port] = 1; + } + return; +} + +function bad_header_stop(port) { + if (port in lastBlockCheckIsFailed) { + if (lastBlockCheckIsFailed[port] >= 5) global.support.sendEmail( + global.config.general.adminEmail, + 'Quering daemon for ' + port + ' port for last block header is back to normal', + `An warning was sent to you indicating that the the worker failed to return the last block header for ${port} port. + The issue seems to be solved now.` + ); + delete lastBlockCheckIsFailed[port]; + } +} + +function monitorNodes() { + global.mysql.query("SELECT blockID, hostname, ip, port FROM pools WHERE last_checkin > date_sub(now(), interval 30 minute)").then(function (rows) { + global.coinFuncs.getPortLastBlockHeader(global.config.daemon.port, function (err, block) { + if (err !== null){ + bad_header_start(global.config.daemon.port); + return; + } + bad_header_stop(); + let top_height = 0; + let is_master_daemon_issue = rows.length > 1 ? true : false; + rows.forEach(function (row) { + if (row.port && row.port != global.config.daemon.port) { + console.error("INTERNAL ERROR: pool node port " + row.port + " do not match master port " + global.config.daemon.port); + is_master_daemon_issue = false; + return; + } + if (top_height < row.blockID) top_height = row.blockID; + if (Math.abs(block.height - row.blockID) > 3) { + global.support.sendEmail(global.config.general.adminEmail, + "Pool server behind in blocks", + "The pool server: " + row.hostname + " with IP: " + row.ip + " is " + (block.height - row.blockID) + " blocks behind for " + row.port + " port" + ); + } else { + is_master_daemon_issue = false; + } + }); + if (is_master_daemon_issue) global.coinFuncs.fixDaemonIssue(block.height, top_height, global.config.daemon.port); + }); + }).catch(function (error) { + console.error("SQL query failed: " + error); + }); +} + +updatePoolStats(); +updatePoolStats('pplns'); +if (global.config.pps.enable === true) updatePoolStats('pps'); +if (global.config.solo.enable === true) updatePoolStats('solo'); +updatePoolInformation(); +updateBlockHeader(); + +monitorNodes(); +setInterval(monitorNodes, 5*60*1000); +setInterval(get_cmc, 15*60*1000, function() {}); + diff --git a/lib/remoteShare.js b/lib/remoteShare.js index 0c564aef..f3cf604e 100644 --- a/lib/remoteShare.js +++ b/lib/remoteShare.js @@ -24,7 +24,7 @@ app.use(function(req, res, next){ // Master/Slave communication Handling function messageHandler(message) { - if (typeof message.shares === "number"){ + if (typeof message.raw_shares === "number"){ shareData.push(message); } } @@ -53,6 +53,15 @@ app.post('/leafApi', function (req, res) { } }); break; + case global.protos.MESSAGETYPE.ALTBLOCK: + global.database.storeAltBlock(msgData.exInt, msgData.msg, function(data){ + if (!data){ + return res.status(400).end(); + } else { + return res.json({'success': true}); + } + }); + break; case global.protos.MESSAGETYPE.INVALIDSHARE: global.database.storeInvalidShare(msgData.msg, function(data){ if (!data){ @@ -73,6 +82,7 @@ app.post('/leafApi', function (req, res) { function storeShares(){ if (Object.keys(shareData).length > 0){ + console.log('Storing ' + Object.keys(shareData).length + ' shares'); global.database.storeBulkShares(shareData); shareData = []; } @@ -102,7 +112,7 @@ if (cluster.isMaster) { workerList.push(worker); }); } else { - app.listen(8000, function () { + app.listen(8000, 'localhost', function () { console.log('Process ' + process.pid + ' is listening to all incoming requests'); }); } diff --git a/lib/remote_comms.js b/lib/remote_comms.js index a91baaa5..faf018bf 100644 --- a/lib/remote_comms.js +++ b/lib/remote_comms.js @@ -10,17 +10,14 @@ function Database() { async.doUntil( function (intCallback) { request.post({url: global.config.general.shareHost, body: task.body, forever: true}, function (error, response, body) { - if (!error) { - return intCallback(null, response.statusCode); - } - return intCallback(null, 0); + return intCallback(null, error ? 0 : response.statusCode); }); }, - function (data) { - return data === 200; + function (data, untilCB) { + return untilCB(null, data === 200); }, function () { - callback(); + return callback(); }); }, require('os').cpus().length*32); @@ -44,6 +41,16 @@ function Database() { process.send({type: 'sendRemote', body: wsData.toString('hex')}); }; + this.storeAltBlock = function (blockId, blockData) { + let wsData = global.protos.WSData.encode({ + msgType: global.protos.MESSAGETYPE.ALTBLOCK, + key: global.config.api.authKey, + msg: blockData, + exInt: blockId + }); + process.send({type: 'sendRemote', body: wsData.toString('hex')}); + }; + this.storeInvalidShare = function (minerData) { let wsData = global.protos.WSData.encode({ msgType: global.protos.MESSAGETYPE.INVALIDSHARE, @@ -55,10 +62,10 @@ function Database() { }; setInterval(function(queue_obj){ - if (global.database.thread_id === '(Master) '){ - console.log(global.database.thread_id + "Queue debug state: " + queue_obj.length() + " items in the queue " + queue_obj.running() + " items being processed"); + if ((queue_obj.length() > 20 || queue_obj.running() > 20) && global.database.thread_id === '(Master) '){ + console.log(global.database.thread_id + "Remote queue state: " + queue_obj.length() + " items in the queue " + queue_obj.running() + " items being processed"); } - }, 5000, this.sendQueue); + }, 30*1000, this.sendQueue); this.initEnv = function(){ diff --git a/lib/support.js b/lib/support.js index 02be5b58..67d6a904 100644 --- a/lib/support.js +++ b/lib/support.js @@ -1,10 +1,11 @@ "use strict"; const CircularBuffer = require('circular-buffer'); +const https = require('https'); const request = require('request'); -const requestJson = require('request-json'); const moment = require('moment'); const debug = require('debug')('support'); const fs = require('fs'); +const sprintf = require("sprintf-js").sprintf; function circularBuffer(size) { let buffer = CircularBuffer(size); @@ -37,84 +38,222 @@ function circularBuffer(size) { return buffer; } -function sendEmail(toAddress, subject, body){ - request.post(global.config.general.mailgunURL + "/messages", { - auth: { - user: 'api', - pass: global.config.general.mailgunKey +// accumulates email notifications up to one hour (email/subject -> body) +let emailAcc = {}; +// last send time of email (email/subject -> time) +let emailLastSendTime = {}; +let lastEmailSendTime; + +function sendEmailReal(toAddress, subject, email_body, retry) { + if (lastEmailSendTime && Date.now() - lastEmailSendTime < 1000) { + setTimeout(sendEmailReal, 1000, toAddress, subject, email_body, retry); + return; + } + lastEmailSendTime = Date.now(); + const body = JSON.stringify({ + from: global.config.general.emailFrom, + to: toAddress, + subject: subject, + text: email_body + }) + "\n"; + request.post(global.config.general.mailgunURL, { + body: body, + agentOptions: { + rejectUnauthorized: global.config.general.mailgunNoCert === true ? false : true }, - form: { - from: global.config.general.emailFrom, - to: toAddress, - subject: subject, - text: body + headers: { + "Content-Type": "application/json", + "Accept": "application/json", + "Content-Length": body.length, + "Connection": "close" } - }, function(err, response, body){ + }, function(err, response, body) { if (!err && response.statusCode === 200) { - console.log("Email sent successfully! Response: " + body); + debug(email_body); + console.log("Email to '" + toAddress + "' was sent successfully! Response: " + body); } else { - console.error("Did not send e-mail successfully! Response: " + body + " Response: "+JSON.stringify(response)); + if (retry) { + console.error("Did not send e-mail to '" + toAddress + "' successfully! Response: " + body + " Response: "+JSON.stringify(response)); + } else { + setTimeout(sendEmailReal, 50*1000, toAddress, subject, email_body, 1); + } } }); } -function jsonRequest(host, port, data, callback, path) { - path = path || 'json_rpc'; - let uri; - if (global.config.rpc.https) { - uri = "https://" + host + ":" + port + "/"; - } else { - uri = "http://" + host + ":" + port + "/"; - } - debug("JSON URI: " + uri + path + " Args: " + JSON.stringify(data)); - let client = requestJson.createClient(uri, {timeout: 300000}); - client.headers["Content-Type"] = "application/json"; - client.headers["Content-Length"] = data.length; - client.headers["Accept"] = "application/json"; - if (global.config.payout.rpcPasswordEnabled && host === global.config.wallet.address && port === global.config.wallet.port){ - fs.readFile(global.config.payout.rpcPasswordPath, 'utf8', function(err, data){ - if (err){ - console.error("RPC password enabled, unable to read the file due to: " + JSON.stringify(err)); - return; - } - let passData = data.split(":"); - client.setBasicAuth(passData[0], passData[1]); - request.post(uri, { - auth:{ - user: passData[0], - pass: passData[1], - sendImmediately: false - }, - data: JSON.stringify(data) - }, function (err, res, body) { - if (err) { - return callback(err); - } - debug("JSON result: " + JSON.stringify(body)); - return callback(body); - }); - }); +function sendEmail(toAddress, subject, body, wallet){ + if (toAddress === global.config.general.adminEmail && !subject.includes("FYI")) { + sendEmailReal(toAddress, subject, body); } else { - client.post(path, data, function (err, res, body) { - if (err) { - return callback(err); - } - debug("JSON result: " + JSON.stringify(body)); - return callback(body); - }); + let reEmail = /^([a-zA-Z0-9_\.-])+@(([a-zA-Z0-9-])+\.)+([a-zA-Z0-9]{2,4})+$/; + if (!reEmail.test(toAddress)) { + debug("Avoid sending email to invalid address '" + toAddress + "'"); + return; + } + let key = toAddress + "\t" + subject; + if (!(key in emailAcc)) { + emailAcc[key] = body; + let time_now = Date.now(); + let is_fast_email = !(key in emailLastSendTime) || time_now - emailLastSendTime[key] > 6*60*60*1000; + emailLastSendTime[key] = time_now; + setTimeout(function(email_address, email_subject, wallet) { + let key2 = email_address + "\t" + email_subject; + let email_body = emailAcc[key2]; + delete emailAcc[key2]; + let emailData = { + wallet: wallet + }; + sendEmailReal(email_address, email_subject, "Hello,\n\n" + email_body + "\n\nThank you,\n" + sprintf(global.config.general.emailSig, emailData)); + }, (is_fast_email ? 5 : 30)*60*1000, toAddress, subject, wallet); + } else { + emailAcc[key] += body; + } } } -function rpc(host, port, method, params, callback) { +function sendEmailAdmin(subject, body){ + sendEmail(global.config.general.adminEmail, subject, body); +} +function jsonRequest(host, port, data, callback, path, timeout) { + let options = { + url: (global.config.rpc.https ? "https://" : "http://") + host + ":" + port + "/" + path, + method: data ? "POST" : "GET", + timeout: timeout, + headers: { + "Content-Type": "application/json", + "Accept": "application/json", + "Connection": "close", + } + }; + if (global.config.daemon.basicAuth) { + options.headers["Authorization"] = global.config.daemon.basicAuth; + } + if (global.config.daemon["X-API-KEY"]) { + options.headers["X-API-KEY"] = global.config.daemon["X-API-KEY"]; + options.headers["api_key"] = global.config.daemon["X-API-KEY"]; + } + + if (data) { + const data2 = typeof data === 'string' ? data : JSON.stringify(data); + options.headers["Content-Length"] = data2.length; + options.body = data2; + } + let reply_fn = function (err, res, body) { + if (err) { + if (typeof(err) === "string") console.error("Error doing " + uri + path + " request: " + err); + return callback(err); + } + let json; + try { + json = JSON.parse(body); + } catch (e) { + debug("JSON parse exception: " + body); + return callback("JSON parse exception: " + body); + } + debug("JSON result: " + JSON.stringify(json)); + return callback(json, res.statusCode); + }; + debug("JSON REQUST: " + JSON.stringify(options)); + request(options, reply_fn); +} + +function rpc(host, port, method, params, callback, timeout) { let data = { id: "0", jsonrpc: "2.0", method: method, params: params }; - return jsonRequest(host, port, data, callback); + return jsonRequest(host, port, data, callback, 'json_rpc', timeout); +} + +function rpc2(host, port, method, params, callback, timeout) { + return jsonRequest(host, port, params, callback, method, timeout); } + +function https_get(url, callback) { + let timer; + let is_callback_called = false; + var req = https.get(url, function(res) { + if (res.statusCode != 200) { + if (timer) clearTimeout(timer); + console.error("URL " + url + ": Result code: " + res.statusCode); + if (!is_callback_called) { + is_callback_called = true; + callback(null); + } + return; + } + let str = ""; + res.on('data', function(d) { str += d; }); + res.on('end', function() { + if (timer) clearTimeout(timer); + let json; + try { + json = JSON.parse(str); + } catch (e) { + console.error("URL " + url + ": JSON parse exception: " + e); + if (!is_callback_called) { + is_callback_called = true; + callback(str); + } + return; + } + if (!is_callback_called) { + is_callback_called = true; + callback(json); + } + return; + }); + res.on('error', function() { + if (timer) clearTimeout(timer); + console.error("URL " + url + ": RESPONSE ERROR!"); + if (!is_callback_called) { + is_callback_called = true; + callback(null); + } + }); + }); + req.on('error', function() { + if (timer) clearTimeout(timer); + console.error("URL " + url + ": REQUEST ERROR!"); + if (!is_callback_called) { + is_callback_called = true; + callback(null); + } + }); + timer = setTimeout(function() { + req.abort(); + console.error("URL " + url + ": TIMEOUT!"); + if (!is_callback_called) { + is_callback_called = true; + callback(null); + } + }, 30*1000); + req.end(); +} + +function getCoinHashFactor(coin, callback) { + global.mysql.query("SELECT item_value FROM config WHERE module = 'daemon' and item = 'coinHashFactor" + coin + "'").then(function (rows) { + if (rows.length != 1) { + console.error("Can't get config.daemon.coinHashFactor" + coin + " value"); + return callback(null); + } + callback(parseFloat(rows[0].item_value)); + }).catch(function (error) { + console.error("SQL query failed: " + error); + return callback(0); + }); +} + +function setCoinHashFactor(coin, coinHashFactor) { + global.mysql.query("UPDATE config SET item_value = ? WHERE module = 'daemon' and item = 'coinHashFactor" + coin + "'", [coinHashFactor]).catch(function (error) { + console.error("SQL query failed: " + error); + }); + global.config.daemon["coinHashFactor" + coin] = coinHashFactor; +} + function formatDate(date) { // Date formatting for MySQL date time fields. return moment(date).format('YYYY-MM-DD HH:mm:ss'); @@ -134,14 +273,6 @@ function decimalToCoin(amount) { return Math.round(amount * global.config.coin.sigDigits); } -function bitcoinDecimalToCoin(amount) { - return Math.round(amount * 100000000); -} - -function bitcoinCoinToDecimal(amount) { - return amount / 100000000; -} - function blockCompare(a, b) { if (a.height < b.height) { return 1; @@ -164,24 +295,49 @@ function tsCompare(a, b) { return 0; } +function port_wallet_ip(port) { + const ip = global.config.wallet["address_" + port.toString()]; + if (ip) return ip; + return global.config.wallet.address; +} + module.exports = function () { return { rpcDaemon: function (method, params, callback) { - rpc(global.config.daemon.address, global.config.daemon.port, method, params, callback); + rpc(global.config.daemon.address, global.config.daemon.port, method, params, callback, 30*1000); + }, + rpcPortDaemon: function (port, method, params, callback) { + rpc(global.config.daemon.address, port, method, params, callback, 30*1000); + }, + rpcPortDaemon2: function (port, method, params, callback) { + rpc2(global.config.daemon.address, port, method, params, callback, 30*1000); }, rpcWallet: function (method, params, callback) { - rpc(global.config.wallet.address, global.config.wallet.port, method, params, callback); + rpc(port_wallet_ip(global.config.wallet.port), global.config.wallet.port, method, params, callback, 30*60*1000); + }, + rpcPortWallet: function (port, method, params, callback) { + rpc(port_wallet_ip(port), port, method, params, callback, 30*60*1000); + }, + rpcPortWallet2: function (port, method, params, callback) { + rpc2(port_wallet_ip(port), port, method, params, callback, 30*60*1000); + }, + rpcPortWalletShort: function (port, method, params, callback) { + rpc(port_wallet_ip(port), port, method, params, callback, 10*1000); + }, + rpcPortWalletShort2: function (port, method, params, callback) { + rpc2(port_wallet_ip(port), port, method, params, callback, 10*1000); }, - jsonRequest: jsonRequest, circularBuffer: circularBuffer, formatDate: formatDate, coinToDecimal: coinToDecimal, decimalToCoin: decimalToCoin, - bitcoinDecimalToCoin: bitcoinDecimalToCoin, - bitcoinCoinToDecimal: bitcoinCoinToDecimal, formatDateFromSQL: formatDateFromSQL, blockCompare: blockCompare, sendEmail: sendEmail, - tsCompare: tsCompare + sendEmailAdmin: sendEmailAdmin, + tsCompare: tsCompare, + getCoinHashFactor: getCoinHashFactor, + setCoinHashFactor: setCoinHashFactor, + https_get: https_get, }; }; diff --git a/lib/worker.js b/lib/worker.js index 8714ca9a..48b69ac5 100644 --- a/lib/worker.js +++ b/lib/worker.js @@ -1,537 +1,449 @@ "use strict"; -const debug = require("debug")("worker"); -const async = require("async"); +const debug = require("debug")("worker"); const sprintf = require("sprintf-js").sprintf; -let threadName = "Worker Server "; let cycleCount = 0; -let lastBlockHash = null; +let hashrate_avg_min = 10; +let stat_change_alert = 0.6; -function updateShareStats() { +let prev_pool_state_time; +let prev_pool_hashrate; +let prev_pool_workers; + +let stats_cache = {}; +let miner_history_update_time = {}; + +let pool_type_str = {}; +pool_type_str[global.protos.POOLTYPE.PPLNS] = 'pplns'; +pool_type_str[global.protos.POOLTYPE.PPS] = 'pps'; +pool_type_str[global.protos.POOLTYPE.SOLO] = 'solo'; + +let identifiers = {}; +let minerSet = {}; +let minerPortSet = {}; +let localMinerCount = {}; +let localStats = {}; +let localPortHashes = {}; +let localTimes = {}; + +let prevMinerSet = {}; +let cache_updates = {}; +let portMinerCount = {}; + +function updateShareStats2(height, callback) { // This is an omni-worker to deal with all things share-stats related // Time based averages are worked out on ring buffers. // Buffer lengths? You guessed it, configured in SQL. // Stats timeouts are 30 seconds, so everything for buffers should be there. - let currentTime = Date.now(); - let activeAddresses = []; - async.waterfall([ - function (callback) { - global.coinFuncs.getLastBlockHeader(function (err, body) { - if (err !== null){ - return callback(err, "Invalid block header"); - } - callback(null, body.height + 1); - }); - }, - function (height, callback) { - let locTime = Date.now() - 600000; - let identifierTime = Date.now() - 1800000; - let localStats = {pplns: 0, pps: 0, solo: 0, prop: 0, global: 0, miners: {}}; - let localMinerCount = {pplns: 0, pps: 0, solo: 0, prop: 0, global: 0}; - let localTimes = { - pplns: locTime, pps: locTime, solo: locTime, prop: locTime, - global: locTime, miners: {} - }; - let minerList = []; - let identifiers = {}; - let loopBreakout = 0; - async.doUntil(function (callback_until) { - let oldestTime = Date.now(); - let loopCount = 0; - let txn = global.database.env.beginTxn({readOnly: true}); - let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); - for (let found = (cursor.goToRange(height) === height); found; found = cursor.goToNextDup()) { - cursor.getCurrentBinary(function (key, share) { // jshint ignore:line - try { - share = global.protos.Share.decode(share); - } catch (e) { - console.error(share); - return; - } - if (share.timestamp < oldestTime) { - oldestTime = share.timestamp; - } - if (share.timestamp <= identifierTime) { - return; - } - let minerID = share.paymentAddress; - if (typeof(share.paymentID) !== 'undefined' && share.paymentID.length > 10) { - minerID = minerID + '.' + share.paymentID; - } - if (minerID in identifiers && identifiers[minerID].indexOf(share.identifier) >= 0) { - loopCount += 1; - } else if (minerID in identifiers) { - identifiers[minerID].push(share.identifier); - } else { - identifiers[minerID] = [share.identifier]; - } - if (share.timestamp <= locTime) { - return; - } - let minerIDWithIdentifier = minerID + "_" + share.identifier; - localStats.global += share.shares; - if (localTimes.global <= share.timestamp) { - localTimes.global = share.timestamp; - } - let minerType; - switch (share.poolType) { - case global.protos.POOLTYPE.PPLNS: - minerType = 'pplns'; - localStats.pplns += share.shares; - if (localTimes.pplns <= share.timestamp) { - localTimes.pplns = share.timestamp; - } - break; - case global.protos.POOLTYPE.PPS: - localStats.pps += share.shares; - minerType = 'pps'; - if (localTimes.pps <= share.timestamp) { - localTimes.pps = share.timestamp; - } - break; - case global.protos.POOLTYPE.SOLO: - localStats.solo += share.shares; - minerType = 'solo'; - if (localTimes.solo <= share.timestamp) { - localTimes.solo = share.timestamp; - } - break; - } - if (minerList.indexOf(minerID) >= 0) { - localStats.miners[minerID] += share.shares; - if (localTimes.miners[minerID] < share.timestamp) { - localTimes.miners[minerID] = share.timestamp; - } - } else { - localMinerCount[minerType] += 1; - localMinerCount.global += 1; - localStats.miners[minerID] = share.shares; - localTimes.miners[minerID] = share.timestamp; - minerList.push(minerID); - } - if (minerList.indexOf(minerIDWithIdentifier) >= 0) { - localStats.miners[minerIDWithIdentifier] += share.shares; - if (localTimes.miners[minerIDWithIdentifier] < share.timestamp) { - localTimes.miners[minerIDWithIdentifier] = share.timestamp; - } - } else { - localStats.miners[minerIDWithIdentifier] = share.shares; - localTimes.miners[minerIDWithIdentifier] = share.timestamp; - minerList.push(minerIDWithIdentifier); - } - }); - } - cursor.close(); - txn.abort(); - return callback_until(null, oldestTime); - }, function (oldestTime) { - height -= 1; - loopBreakout += 1; - if (loopBreakout > 60) { - return true; - } - return oldestTime <= identifierTime; - }, function (err) { - // todo: Need to finish parsing the cached data into caches for caching purproses. - let globalMinerList = global.database.getCache('minerList'); - if (globalMinerList === false) { - globalMinerList = []; - } - let cache_updates = {}; - // pplns: 0, pps: 0, solo: 0, prop: 0, global: 0 - ['pplns', 'pps', 'solo', 'prop', 'global'].forEach(function (key) { - let cachedData = global.database.getCache(key + "_stats"); - if (cachedData !== false) { - cachedData.hash = Math.floor(localStats[key] / 600); - cachedData.lastHash = localTimes[key]; - cachedData.minerCount = localMinerCount[key]; - if (!cachedData.hasOwnProperty("hashHistory")) { - cachedData.hashHistory = []; - cachedData.minerHistory = []; - } - if (cycleCount === 0) { - cachedData.hashHistory.unshift({ts: currentTime, hs: cachedData.hash}); - if (cachedData.hashHistory.length > global.config.general.statsBufferLength) { - while (cachedData.hashHistory.length > global.config.general.statsBufferLength) { - cachedData.hashHistory.pop(); - } - } - cachedData.minerHistory.unshift({ts: currentTime, cn: cachedData.minerCount}); - if (cachedData.minerHistory.length > global.config.general.statsBufferLength) { - while (cachedData.minerHistory.length > global.config.general.statsBufferLength) { - cachedData.minerHistory.pop(); - } - } - } - } else { - cachedData = { - hash: Math.floor(localStats[key] / 600), - totalHashes: 0, - lastHash: localTimes[key], - minerCount: localMinerCount[key], - hashHistory: [{ts: currentTime, hs: cachedData.hash}], - minerHistory: [{ts: currentTime, cn: cachedData.hash}] - }; - } - cache_updates[key + "_stats"] = cachedData; - }); - minerList.forEach(function (miner) { - if (globalMinerList.indexOf(miner) === -1) { - globalMinerList.push(miner); - } - if (miner.indexOf('_') === -1){ - activeAddresses.push(miner); - } - let cachedData = global.database.getCache(miner); - if (cachedData !== false) { - cachedData.hash = Math.floor(localStats.miners[miner] / 600); - cachedData.lastHash = localTimes.miners[miner]; - if (!cachedData.hasOwnProperty("hashHistory")) { - cachedData.hashHistory = []; - } - if (cycleCount === 0){ - cachedData.hashHistory.unshift({ts: currentTime, hs: cachedData.hash}); - if (cachedData.hashHistory.length > global.config.general.statsBufferLength) { - while (cachedData.hashHistory.length > global.config.general.statsBufferLength) { - cachedData.hashHistory.pop(); - } - } - } - } else { - cachedData = { - hash: Math.floor(localStats.miners[miner] / 600), - totalHashes: 0, - lastHash: localTimes.miners[miner], - hashHistory: [{ts: currentTime, hs: cachedData.hash}], - goodShares: 0, - badShares: 0 - }; - } - cache_updates[miner] = cachedData; - }); - globalMinerList.forEach(function (miner) { - if (minerList.indexOf(miner) === -1) { - let minerStats = global.database.getCache(miner); - if (minerStats.hash !== 0) { - console.log("Removing: " + miner + " as an active miner from the cache."); - if (miner.indexOf('_') > -1) { - // This is a worker case. - let address_parts = miner.split('_'); - let address = address_parts[0]; - let worker = address_parts[1]; - global.mysql.query("SELECT email FROM users WHERE username = ? AND enable_email IS true limit 1", [address]).then(function (rows) { - if (rows.length === 0) { - return; - } - // toAddress, subject, body - let emailData = { - worker: worker, - timestamp: global.support.formatDate(Date.now()), - poolEmailSig: global.config.general.emailSig - }; - global.support.sendEmail(rows[0].email, - sprintf(global.config.email.workerNotHashingSubject, emailData), - sprintf(global.config.email.workerNotHashingBody, emailData)); - }); - } - minerStats.hash = 0; - cache_updates[miner] = minerStats; - } - } - }); - Object.keys(identifiers).forEach(function (key) { - cache_updates[key + '_identifiers'] = identifiers[key]; - }); - cache_updates.minerList = globalMinerList; - global.database.bulkSetCache(cache_updates); - callback(null); - }); - } - ], function (err, result) { - cycleCount += 1; - if (cycleCount === 6){ - cycleCount = 0; - } - }); - setTimeout(updateShareStats, 10000); -} + const currentTime = Date.now(); + // power to ensure we can keep up to global.config.general.statsBufferHours in global.config.general.statsBufferLength array + // here N = log(history_power, global.config.general.statsBufferLength) is number of attemps required on average to remove top left history point (the oldest one) + // we just select history_power so that is till happen on global.config.general.statsBufferHours * 60 attemps on average + const history_power = Math.log(global.config.general.statsBufferLength) / Math.log(global.config.general.statsBufferHours * 60); -function updatePoolStats(poolType) { - let cache; - if (typeof(poolType) !== 'undefined') { - cache = global.database.getCache(poolType + "_stats"); - } else { - cache = global.database.getCache("global_stats"); - } - async.series([ - function (callback) { - debug(threadName + "Checking Influx for last 10min avg for pool stats"); - return callback(null, cache.hash || 0); - }, - function (callback) { - debug(threadName + "Checking Influx for last 10min avg for miner count for pool stats"); - return callback(null, cache.minerCount || 0); - }, - function (callback) { - debug(threadName + "Checking Influx for last 10min avg for miner count for pool stats"); - return callback(null, cache.totalHashes || 0); - }, - function (callback) { - debug(threadName + "Checking MySQL for last block find time for pool stats"); - let cacheData = global.database.getBlockList(poolType); - if (cacheData.length === 0) { - return callback(null, 0); + console.log("Starting stats collection for " + height + " height (history power: " + history_power + ")"); + + const locTime = currentTime - (hashrate_avg_min*60*1000); + const identifierTime = currentTime - (2*hashrate_avg_min*60*1000); + + let minerCount = 0; + + identifiers = {}; + minerSet = {}; + minerPortSet = {}; + localMinerCount = { pplns: 0, pps: 0, solo: 0, prop: 0, global: 0 }; + localStats = { pplns: 0, pps: 0, solo: 0, prop: 0, global: 0, miners: {}, miners2: {} }; + localPortHashes = {}; + localTimes = { pplns: locTime, pps: locTime, solo: locTime, prop: locTime, global: locTime, miners: {} }; + + let loopBreakout = 0; + let oldestTime = currentTime; + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + + do { + let count = 0; + for (let found = cursor.goToRange(height) === height; found; ++ count, found = cursor.goToNextDup()) cursor.getCurrentBinary(function (key, share) { // jshint ignore:line + try { + share = global.protos.Share.decode(share); + } catch (e) { + console.error(share); + return; } - return callback(null, Math.floor(cacheData[0].ts / 1000)); - }, - function (callback) { - debug(threadName + "Checking MySQL for last block find time for pool stats"); - let cacheData = global.database.getBlockList(poolType); - if (cacheData.length === 0) { - return callback(null, 0); + if (share.timestamp < oldestTime) oldestTime = share.timestamp; + if (share.timestamp <= identifierTime) return; + + const minerID = typeof(share.paymentID) !== 'undefined' && share.paymentID.length > 10 + ? share.paymentAddress + '.' + share.paymentID : share.paymentAddress; + + const identifier = share.identifier; + + if (minerID in identifiers) { + if (identifiers[minerID].indexOf(identifier) < 0) { + identifiers[minerID].push(identifier); + ++ minerCount; + } + } else { + identifiers[minerID] = [identifier]; + ++ minerCount; } - return callback(null, cacheData[0].height); - }, - function (callback) { - debug(threadName + "Checking MySQL for block count for pool stats"); - return callback(null, global.database.getBlockList(poolType).length); - }, - function (callback) { - debug(threadName + "Checking MySQL for total miners paid"); - if (typeof(poolType) !== 'undefined') { - global.mysql.query("SELECT payment_address, payment_id FROM payments WHERE pool_type = ? group by payment_address, payment_id", [poolType]).then(function (rows) { - return callback(null, rows.length); - }); + + if (share.timestamp <= locTime) return; + + const minerIDWithIdentifier = minerID + "_" + identifier; + const shares2 = share.shares2; + localStats.global += shares2; + if (localTimes.global < share.timestamp) localTimes.global = share.timestamp; + const minerType = pool_type_str[share.poolType]; + if (!minerType) { + console.error("Wrong share pool type found: " + share.poolType); + return; + } + localStats[minerType] += shares2; + if (localTimes[minerType] < share.timestamp) localTimes[minerType] = share.timestamp; + + const port = typeof(share.port) !== 'undefined' && share.port ? share.port : global.config.daemon.port; + if (port in localPortHashes) localPortHashes[port] += share.raw_shares; + else localPortHashes[port] = share.raw_shares; + + if (!shares2) return; // use virtual shares from child block mining only for global pool stats + + if (minerID in minerPortSet) { + localStats.miners[minerID] += share.raw_shares; + localStats.miners2[minerID] += shares2; + if (localTimes.miners[minerID] < share.timestamp) localTimes.miners[minerID] = share.timestamp; } else { - global.mysql.query("SELECT payment_address, payment_id FROM payments group by payment_address, payment_id").then(function (rows) { - return callback(null, rows.length); - }); + ++ localMinerCount[minerType]; + ++ localMinerCount.global; + localStats.miners[minerID] = share.raw_shares; + localStats.miners2[minerID] = shares2; + localTimes.miners[minerID] = share.timestamp; + minerSet[minerID] = 1; + minerPortSet[minerID] = port; } - }, - function (callback) { - debug(threadName + "Checking MySQL for total transactions count"); - if (typeof(poolType) !== 'undefined') { - global.mysql.query("SELECT distinct(transaction_id) from payments WHERE pool_type = ?", [poolType]).then(function (rows) { - return callback(null, rows.length); - }); + + if (minerIDWithIdentifier in minerSet) { + localStats.miners[minerIDWithIdentifier] += share.raw_shares; + localStats.miners2[minerIDWithIdentifier] += shares2; + if (localTimes.miners[minerIDWithIdentifier] < share.timestamp) localTimes.miners[minerIDWithIdentifier] = share.timestamp; } else { - global.mysql.query("SELECT count(id) as txn_count FROM transactions").then(function (rows) { - if (typeof(rows[0]) !== 'undefined') { - return callback(null, rows[0].txn_count); - } else { - return callback(null, 0); - } - }); + localStats.miners[minerIDWithIdentifier] = share.raw_shares; + localStats.miners2[minerIDWithIdentifier] = shares2; + localTimes.miners[minerIDWithIdentifier] = share.timestamp; + minerSet[minerIDWithIdentifier] = 1; } - }, - function (callback) { - debug(threadName + "Checking Influx for last 10min avg for miner count for pool stats"); - return callback(null, cache.roundHashes || 0); - } - ], function (err, result) { - if (typeof(poolType) === 'undefined') { - poolType = 'global'; - } - global.database.setCache('pool_stats_' + poolType, { - hashRate: result[0], - miners: result[1], - totalHashes: result[2], - lastBlockFoundTime: result[3] || 0, - lastBlockFound: result[4] || 0, - totalBlocksFound: result[5] || 0, - totalMinersPaid: result[6] || 0, - totalPayments: result[7] || 0, - roundHashes: result[8] || 0 }); - }); -} + debug("On " + height + " height iterated " + count + " elements"); + } while (++loopBreakout <= 60 && --height >= 0 && oldestTime > identifierTime); + cursor.close(); + txn.abort(); -function updatePoolPorts(poolServers) { - debug(threadName + "Updating pool ports"); - let local_cache = {global: []}; - let portCount = 0; - global.mysql.query("select * from ports where hidden = 0 and lastSeen >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { - rows.forEach(function (row) { - portCount += 1; - if (!local_cache.hasOwnProperty(row.port_type)) { - local_cache[row.port_type] = []; + debug("Share loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + prevMinerSet = global.database.getCache('minerSet'); + if (prevMinerSet === false) prevMinerSet = minerSet; + cache_updates = {}; + // pplns: 0, pps: 0, solo: 0, prop: 0, global: 0 + ['pplns', 'pps', 'solo', 'prop', 'global'].forEach(function (key) { + const hash = localStats[key] / (hashrate_avg_min*60); + const lastHash = localTimes[key]; + const minerCount = localMinerCount[key]; + let cachedData = global.database.getCache(key + "_stats"); + if (cachedData !== false) { + cachedData.hash = hash; + cachedData.lastHash = lastHash; + cachedData.minerCount = minerCount; + if (!cachedData.hasOwnProperty("hashHistory")) { + cachedData.hashHistory = []; + cachedData.minerHistory = []; } - local_cache[row.port_type].push({ - host: poolServers[row.pool_id], - port: row.network_port, - difficulty: row.starting_diff, - description: row.description, - miners: row.miners - }); - if (portCount === rows.length) { - let local_counts = {}; - let port_diff = {}; - let port_miners = {}; - let pool_type_count = 0; - let localPortInfo = {}; - for (let pool_type in local_cache) { // jshint ignore:line - pool_type_count += 1; - local_cache[pool_type].forEach(function (portData) { // jshint ignore:line - if (!local_counts.hasOwnProperty(portData.port)) { - local_counts[portData.port] = 0; - } - if (!port_diff.hasOwnProperty(portData.port)) { - port_diff[portData.port] = portData.difficulty; - } - if (!port_miners.hasOwnProperty(portData.port)) { - port_miners[portData.port] = 0; - } - if (port_diff[portData.port] === portData.difficulty) { - local_counts[portData.port] += 1; - port_miners[portData.port] += portData.miners; - } - localPortInfo[portData.port] = portData.description; - if (local_counts[portData.port] === Object.keys(poolServers).length) { - local_cache.global.push({ - host: { - blockID: local_cache[pool_type][0].host.blockID, - blockIDTime: local_cache[pool_type][0].host.blockIDTime, - hostname: global.config.pool.geoDNS, - }, - port: portData.port, - pool_type: pool_type, - difficulty: portData.difficulty, - miners: port_miners[portData.port], - description: localPortInfo[portData.port] - }); - } - }); - if (pool_type_count === Object.keys(local_cache).length) { - debug(threadName + "Sending the following to the workers: " + JSON.stringify(local_cache)); - global.database.setCache('poolPorts', local_cache); + if (cycleCount === 0) { + cachedData.hashHistory.unshift({ts: currentTime, hs: cachedData.hash}); + if (cachedData.hashHistory.length > global.config.general.statsBufferLength) { + while (cachedData.hashHistory.length > global.config.general.statsBufferLength) { + cachedData.hashHistory.pop(); + } + } + cachedData.minerHistory.unshift({ts: currentTime, cn: cachedData.minerCount}); + if (cachedData.minerHistory.length > global.config.general.statsBufferLength) { + while (cachedData.minerHistory.length > global.config.general.statsBufferLength) { + cachedData.minerHistory.pop(); } } } - }); - }); -} - -function updatePoolInformation() { - let local_cache = {}; - debug(threadName + "Updating pool information"); - global.mysql.query("select * from pools where last_checkin >= NOW() - INTERVAL 10 MINUTE").then(function (rows) { - rows.forEach(function (row) { - local_cache[row.id] = { - ip: row.ip, - blockID: row.blockID, - blockIDTime: global.support.formatDateFromSQL(row.blockIDTime), - hostname: row.hostname + } else { + cachedData = { + hash: hash, + totalHashes: 0, + lastHash: lastHash, + minerCount: minerCount, + hashHistory: [{ts: currentTime, hs: hash}], + minerHistory: [{ts: currentTime, cn: minerCount}] }; - if (Object.keys(local_cache).length === rows.length) { - global.database.setCache('poolServers', local_cache); - updatePoolPorts(local_cache); - } - }); + } + cache_updates[key + "_stats"] = cachedData; }); -} + for (let port in localPortHashes) localPortHashes[port] = localPortHashes[port] / (hashrate_avg_min*60); + cache_updates["port_hash"] = localPortHashes; + let history_update_count = 0; -function updateBlockHeader() { - // Todo: Implement within the coins/.js file. - global.support.rpcDaemon('getlastblockheader', [], function (body) { - if (typeof body.error !== 'undefined'){ - return console.error(`Issue getting last block header: ${JSON.stringify(body)}`); - } - if (body.result && body.result.block_header.hash !== lastBlockHash) { - lastBlockHash = body.result.block_header.hash; - global.database.setCache('networkBlockInfo', { - difficulty: body.result.block_header.difficulty, - hash: body.result.block_header.hash, - height: body.result.block_header.height, - value: body.result.block_header.reward, - ts: body.result.block_header.timestamp - }); - } else if (body.result.block_header.hash === lastBlockHash) { - console.log("Block headers identical to historical header. Ignoring"); + for (let miner in minerSet) { + let stats; + let keyStats = "stats:" + miner; + let keyHistory = "history:" + miner; + + if (miner in stats_cache) { + stats = stats_cache[miner]; } else { - console.error("GetLastBlockHeader Error during block header update"); + stats = global.database.getCache(keyStats); + if (!stats) stats = {}; + let history_stats = global.database.getCache(keyHistory); + if (history_stats) { + stats.hashHistory = history_stats.hashHistory; + } else { + stats.hashHistory = []; + } } - }); -} -function updateWalletStats() { - async.waterfall([ - function (callback) { - // Todo: Implement within the coins/.js file. - global.support.rpcWallet('getbalance', [], function (body) { - if (body.result) { - return callback(null, { - balance: body.result.balance, - unlocked: body.result.unlocked_balance, - ts: Date.now() - }); - } else { - return callback(true, "Unable to process balance"); - } - }); - }, - function (state, callback) { - // Todo: Implement within the coins/.js file. - global.support.rpcWallet('getheight', [], function (body) { - if (body.result) { - state.height = body.result.height; - return callback(null, state); - } else if (typeof body.error !== 'undefined' && body.error.message === 'Method not found') { - state.height = 0; - return callback(null, state); - } else { - return callback(true, "Unable to get current wallet height"); + stats.hash = localStats.miners[miner] / (hashrate_avg_min*60); + stats.hash2 = localStats.miners2[miner] / (hashrate_avg_min*60); + stats.lastHash = localTimes.miners[miner]; + cache_updates[keyStats] = { hash: stats.hash, hash2: stats.hash2, lastHash: stats.lastHash }; + + if (cycleCount === 0) { + stats.hashHistory.unshift({ts: currentTime, hs: stats.hash, hs2: stats.hash2}); + if (stats.hashHistory.length > global.config.general.statsBufferLength) { + const is_worker = miner.indexOf('_') >= 0; + while (stats.hashHistory.length > global.config.general.statsBufferLength) { + if (is_worker) { + stats.hashHistory.pop(); + } else { + const last_index = stats.hashHistory.length - 1; + if ((currentTime - stats.hashHistory[last_index].ts) / 1000 / 3600 > global.config.general.statsBufferHours) { + stats.hashHistory.pop(); + } else { + // here we remove larger indexes (that are more distant in time) with more probability + const index_to_remove = (last_index * (1 - Math.pow(Math.random(), history_power))).toFixed(); + stats.hashHistory.splice(index_to_remove, 1); + } + } } - }); - } - ], function (err, results) { - if (err) { - return console.error("Unable to get wallet stats: " + results); + } + if ( stats.hashHistory.length < global.config.general.statsBufferLength || + !(miner in miner_history_update_time) || + (history_update_count < 5000 && currentTime - miner_history_update_time[miner] > 10*60*1000) + ) { + cache_updates[keyHistory] = { hashHistory: stats.hashHistory }; + miner_history_update_time[miner] = currentTime; + ++ history_update_count; + } } - global.database.setCache('walletStateInfo', results); - let history = global.database.getCache('walletHistory'); - if (history === false) { - history = []; + + stats_cache[miner] = stats; + } + + debug("History loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + + // remove old workers + for (let miner in prevMinerSet) { + if (miner in minerSet) continue; // we still have this miner in current set + //debug("Removing: " + miner + " as an active miner from the cache."); + let minerStats = global.database.getCache(miner); + if (!minerStats) continue; + minerStats.hash = 0; + cache_updates[miner] = minerStats; + if (miner.indexOf('_') <= -1) continue; + + // This is a worker case. + const address_parts = miner.split(/_(.+)/); + const worker = address_parts[1]; + if (typeof(worker) !== 'undefined' && !worker.includes('silent')) { + if (!(miner in workers_stopped_hashing_time)) { + workers_stopped_hashing_time[miner] = currentTime; + setTimeout(delayed_send_worker_stopped_hashing_email, 10*60*1000, miner, currentTime); + } } - history.unshift(results); - history = history.sort(global.support.tsCompare); - if (history.length > global.config.general.statsBufferLength) { - while (history.length > global.config.general.statsBufferLength) { - history.pop(); + } + + debug("Old worker loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + + // find new workers + for (let miner in minerSet) { + if (miner in prevMinerSet) continue; // we still have this miner in previous set + //debug("Adding: " + miner + " as an active miner to the cache."); + if (miner.indexOf('_') <= -1) continue; + + // This is a worker case. + const address_parts = miner.split(/_(.+)/); + const worker = address_parts[1]; + if (typeof(worker) !== 'undefined' && !worker.includes('silent')) { + workers_started_hashing_time[miner] = currentTime; + if (miner in workers_stopped_hashing_email_time) { + delete workers_stopped_hashing_time[miner]; + delete workers_stopped_hashing_email_time[miner]; + const address = address_parts[0]; + get_address_email(address, function (email) { + send_worker_started_hashing_email(miner, email, currentTime); + }); } } - global.database.setCache('walletHistory', history); + } + + debug("New worker loop: " + ((Date.now() - currentTime) / 1000) + " seconds"); + + Object.keys(identifiers).forEach(function (key) { + cache_updates['identifiers:' + key] = identifiers[key]; }); + portMinerCount = {}; + for (let miner in minerPortSet) { + const port = minerPortSet[miner]; + if (port in portMinerCount) ++ portMinerCount[port]; + else portMinerCount[port] = 1; + } + cache_updates.portMinerCount = portMinerCount; + cache_updates.minerSet = minerSet; + const db_write_start_time = Date.now(); + try { + global.database.bulkSetCache(cache_updates); + } catch (e) { + console.error("Can't write to pool DB: " + e); + global.support.sendEmail(global.config.general.adminEmail, "FYI: Pool DB is overflowed!", "Can't wite to pool DB: " + e); + } + cache_updates = {}; + + let pool_hashrate = localStats.global / (hashrate_avg_min*60); + let pool_workers = minerCount; + console.log("Processed " + minerCount + " workers (" + history_update_count + " history) for " + + ((Date.now() - currentTime) / 1000) + " seconds (" + ((Date.now() - db_write_start_time) / 1000) + " seconds DB write). " + + "Pool hashrate is: " + pool_hashrate + ); + if (!prev_pool_state_time || currentTime - prev_pool_state_time > hashrate_avg_min*60*1000) { + let pool_hashrate_ratio = prev_pool_hashrate ? pool_hashrate / prev_pool_hashrate : 1; + let pool_workers_ratio = prev_pool_workers ? pool_workers / prev_pool_workers : 1; + if (pool_hashrate_ratio < (1-stat_change_alert) || pool_hashrate_ratio > (1+stat_change_alert) || + pool_workers_ratio < (1-stat_change_alert) || pool_workers_ratio > (1+stat_change_alert)) { + global.support.sendEmail(global.config.general.adminEmail, + "FYI: Pool hashrate/workers changed significantly", + "Pool hashrate changed from " + prev_pool_hashrate + " to " + pool_hashrate + " (" + pool_hashrate_ratio + ")\n" + + "Pool number of workers changed from " + prev_pool_workers + " to " + pool_workers + " (" + pool_workers_ratio + ")\n" + ); + } + prev_pool_hashrate = pool_hashrate; + prev_pool_workers = pool_workers; + prev_pool_state_time = currentTime; + } + return callback(); +} + +function updateShareStats() { + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err !== null){ + return setTimeout(updateShareStats, 20*1000); + } + updateShareStats2(body.height + 1, function() { + if (++cycleCount === 6) cycleCount = 0; + setTimeout(updateShareStats, 20*1000); + }); + }); } -function monitorNodes() { - global.mysql.query("SELECT blockID, hostname, ip FROM pools WHERE last_checkin > date_sub(now(), interval 30 minute)").then(function (rows) { - global.coinFuncs.getLastBlockHeader(function (err, block) { - if (err !== null){ - console.error("Issue in getting block header. Skipping node monitor"); +// cached email of specific address +let minerEmail = {}; +// time of last SQL check for specific address +let minerEmailTime = {}; + +// worker name -> time +let workers_started_hashing_time = {}; +let workers_stopped_hashing_time = {}; +let workers_stopped_hashing_email_time = {}; + +function get_address_email(address, callback) { + let currentTime = Date.now(); + if (!(address in minerEmailTime) || currentTime - minerEmailTime[address] > 10*60*1000) { + minerEmailTime[address] = currentTime; + minerEmail[address] = null; + global.mysql.query("SELECT email FROM users WHERE username = ? AND enable_email IS true limit 1", [address]).then(function (rows) { + if (rows.length === 0) { + delete minerEmail[address]; return; + } else { + minerEmail[address] = rows[0].email; } - rows.forEach(function (row) { - if (row.blockID < block.height - 3) { - global.support.sendEmail(global.config.general.adminEmail, "Pool server behind in blocks", "The pool server: "+row.hostname+" with IP: "+row.ip+" is "+block.height - row.blockID+ " blocks behind"); - } - } - ); + return callback(minerEmail[address]); + }).catch(function (error) { + console.error("Can't get email address for " + address + ": " + error.message); + return; }); + } else if (address in minerEmail) { + if (minerEmail[address] === null) { // not yet ready (retry again in 10 secs) + if (currentTime - minerEmailTime[address] < 5*1000) return setTimeout(get_address_email, 10*1000, address, callback); + } else { + return callback(minerEmail[address]); + } + } +} + +function send_worker_started_hashing_email(miner, email, currentTime) { + let address_parts = miner.split(/_(.+)/); + let address = address_parts[0]; + let worker = address_parts[1]; + // toAddress, subject, body + let emailData = { + worker: worker, + timestamp: global.support.formatDate(currentTime), + poolEmailSig: global.config.general.emailSig + }; + global.support.sendEmail(email, + sprintf(global.config.email.workerStartHashingSubject, emailData), + sprintf(global.config.email.workerStartHashingBody, emailData), + address + ); +} + +function delayed_send_worker_stopped_hashing_email(miner, currentTime) { + if (miner in workers_started_hashing_time && Date.now() - workers_started_hashing_time[miner] <= 10*60*1000) { + delete workers_started_hashing_time[miner]; + return; + } + + delete workers_started_hashing_time[miner]; + + const address_parts = miner.split(/_(.+)/); + const address = address_parts[0]; + + get_address_email(address, function (email) { + workers_stopped_hashing_email_time[miner] = Date.now(); + const worker = address_parts[1]; + + // toAddress, subject, body + const emailData = { + worker: worker, + timestamp: global.support.formatDate(currentTime), + poolEmailSig: global.config.general.emailSig + }; + global.support.sendEmail(email, + sprintf(global.config.email.workerNotHashingSubject, emailData), + sprintf(global.config.email.workerNotHashingBody, emailData), + address + ); }); } + +global.support.sendEmail(global.config.general.adminEmail, "Restarting worker module", "Restarted worker module!"); + updateShareStats(); -updateBlockHeader(); -updatePoolStats(); -updatePoolInformation(); -updateWalletStats(); -monitorNodes(); -setInterval(updateBlockHeader, 10000); -setInterval(updatePoolStats, 5000); -setInterval(updatePoolStats, 5000, 'pplns'); -setInterval(updatePoolStats, 5000, 'pps'); -setInterval(updatePoolStats, 5000, 'solo'); -setInterval(updatePoolInformation, 5000); -setInterval(updateWalletStats, 60000); -setInterval(monitorNodes, 300000); \ No newline at end of file +// clean caches from time to time +setInterval(function() { + console.log("Cleaning caches (" + Object.keys(stats_cache).length + " stats, " + Object.keys(miner_history_update_time).length + " histories)"); + const currentTime = Date.now(); + let stats_cache2 = {}; + for (let miner in stats_cache) { + if (miner in miner_history_update_time && currentTime - miner_history_update_time[miner] < 60*60*1000) { + stats_cache2[miner] = stats_cache[miner]; + } + } + stats_cache = stats_cache2; + console.log("After cleaning: " + Object.keys(stats_cache).length + " stats left"); + miner_history_update_time = {}; +}, 2*60*60*1000); diff --git a/manage_scripts/altblock_add.js b/manage_scripts/altblock_add.js new file mode 100644 index 00000000..e19b27fe --- /dev/null +++ b/manage_scripts/altblock_add.js @@ -0,0 +1,63 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.timestamp) { + console.error("Please specify altblock time"); + process.exit(1); +} +const timestamp = argv.timestamp; + +if (!argv.body) { + console.error("Please specify altblock body"); + process.exit(1); +} +const body = argv.body; +let body2; + +try { body2 = JSON.parse(body); } catch(e) { + console.error("Can't parse altblock body: " + body); + process.exit(1); +} + +require("../init_mini.js").init(function() { + const body3 = { + "hash": body2.hash, + "difficulty": body2.difficulty, + "port": body2.port, + "height": body2.height, + "value": body2.value, + "anchor_height": body2.anchor_height, + "timestamp": timestamp * 1000, + "shares": body2.shares || body2.difficulty, + "poolType": body2.poolType || 0, + "unlocked": body2.unlocked || false, + "valid": body2.valid || true, + "pay_value": body2.pay_value || 0, + "pay_stage": body2.pay_stage || "", + "pay_status": body2.pay_status || "" + }; + if (typeof (body3.hash) === 'undefined' || + typeof (body3.difficulty) === 'undefined' || + typeof (body3.shares) === 'undefined' || + typeof (body3.timestamp) === 'undefined' || + typeof (body3.poolType) === 'undefined' || + typeof (body3.unlocked) === 'undefined' || + typeof (body3.valid) === 'undefined' || + typeof (body3.port) === 'undefined' || + typeof (body3.height) === 'undefined' || + typeof (body3.anchor_height) === 'undefined' || + typeof (body3.value) === 'undefined' || + typeof (body3.pay_value) === 'undefined' || + typeof (body3.pay_stage) === 'undefined' || + typeof (body3.pay_status) === 'undefined') { + console.error("Altblock body is invalid: " + JSON.stringify(body3)); + process.exit(1); + } + const body4 = global.protos.AltBlock.encode(body3); + let txn = global.database.env.beginTxn(); + txn.putBinary(global.database.altblockDB, timestamp, body4); + txn.commit(); + console.log("Altblock with " + timestamp + " timestamp added! Exiting!"); + process.exit(0); +}); diff --git a/manage_scripts/altblock_add_auto.js b/manage_scripts/altblock_add_auto.js new file mode 100644 index 00000000..2df93b6e --- /dev/null +++ b/manage_scripts/altblock_add_auto.js @@ -0,0 +1,69 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +if (!argv.hash) { + console.error("Please specify hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + global.coinFuncs.getLastBlockHeader(function (err, last_block_body) { + if (err !== null){ + console.error("Can't get last block info"); + process.exit(0); + } + global.coinFuncs.getPortAnyBlockHeaderByHash(port, hash, true, function (err_header, body_header) { + if (err_header) { + console.error("Can't get block info"); + console.error("err:" + JSON.stringify(err_header)); + console.error("body:" + JSON.stringify(body_header)); + process.exit(0); + } + if (!body_header.timestamp) body_header.timestamp = body_header.time; + if (!body_header.timestamp) body_header.timestamp = body_header.mediantime; + if (!body_header.timestamp) { + console.error("Can't get block timestamp: " + JSON.stringify(body_header)); + process.exit(0); + } + if ((Date.now() / 1000) < body_header.timestamp) body_header.timestamp = parseInt(body_header.timestamp / 1000); + if (!body_header.difficulty) { + console.error("Can't get block difficilty: " + JSON.stringify(body_header)); + process.exit(0); + } + if (!body_header.height) { + console.error("Can't get block height: " + JSON.stringify(body_header)); + process.exit(0); + } + body_header.difficulty = parseInt(body_header.difficulty); + body_header.timestamp = parseInt(body_header.timestamp); + global.database.storeAltBlock(body_header.timestamp, global.protos.AltBlock.encode({ + hash: hash, + difficulty: body_header.difficulty, + shares: 0, + timestamp: body_header.timestamp * 1000, + poolType: global.protos.POOLTYPE.PPLNS, + unlocked: false, + valid: true, + port: port, + height: body_header.height, + anchor_height: last_block_body.height + }), function(data){ + if (!data){ + console.error("Block not stored"); + } else { + console.log("Block with " + port + " port and " + hash + " stored"); + } + process.exit(0); + }); + }); + }); +}); + diff --git a/manage_scripts/altblock_change_stage.js b/manage_scripts/altblock_change_stage.js new file mode 100644 index 00000000..bbd60b1f --- /dev/null +++ b/manage_scripts/altblock_change_stage.js @@ -0,0 +1,37 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2), { '--': true }); + +if (!argv.stage) { + console.error("Please specify new stage value"); + process.exit(1); +} +const stage = argv.stage; + +let hashes = {}; +for (const h of argv['--']) { + hashes[h] = 1; +} + +require("../init_mini.js").init(function() { + let changed = 0; + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash in hashes) { + console.log("Found altblock with " + blockData.hash + " hash"); + blockData.pay_stage = stage; + console.log("Put \"" + blockData.pay_stage + "\" stage to block"); + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + console.log("Changed altblock"); + changed = 1; + } + }); + } + cursor.close(); + txn.commit(); + if (!changed) console.log("Not found altblocks with specified hashes"); + process.exit(0); +}); \ No newline at end of file diff --git a/manage_scripts/altblock_del.js b/manage_scripts/altblock_del.js new file mode 100644 index 00000000..d884296e --- /dev/null +++ b/manage_scripts/altblock_del.js @@ -0,0 +1,17 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.timestamp) { + console.error("Please specify altblock time"); + process.exit(1); +} +const timestamp = argv.timestamp; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + txn.del(global.database.altblockDB, timestamp); + txn.commit(); + console.log("Altblock with " + timestamp + " timestamp removed! Exiting!"); + process.exit(0); +}); diff --git a/manage_scripts/altblock_fix_raw_reward.js b/manage_scripts/altblock_fix_raw_reward.js new file mode 100644 index 00000000..6be4a892 --- /dev/null +++ b/manage_scripts/altblock_fix_raw_reward.js @@ -0,0 +1,45 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.hash) { + console.error("Please specify altblock hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let is_found = 0; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (!is_found && blockData.hash === hash) { + console.log("Found altblock with " + blockData.hash + " hash"); + is_found = 1; + global.coinFuncs.getPortAnyBlockHeaderByHash(blockData.port, argv.hash, false, function (err, body) { + if (err) { + cursor.close(); + txn.commit(); + console.error("Can't get block header"); + process.exit(1); + } + console.log("Changing raw block reward from " + blockData.value + " to " + body.reward); + blockData.value = body.reward; + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + txn.commit(); + cursor.close(); + console.log("Changed altblock"); + process.exit(0); + }); + } + }); + } + if (!is_found) { + cursor.close(); + txn.commit(); + console.log("Not found altblock with " + hash + " hash"); + process.exit(1); + } +}); diff --git a/manage_scripts/altblock_pay_manually.js b/manage_scripts/altblock_pay_manually.js new file mode 100644 index 00000000..fdd3b8cd --- /dev/null +++ b/manage_scripts/altblock_pay_manually.js @@ -0,0 +1,40 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.hash) { + console.error("Please specify altblock hash"); + process.exit(1); +} +const hash = argv.hash; + +if (!argv.pay) { + console.error("Please specify pay value in main currency"); + process.exit(1); +} +const pay = argv.pay; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash === hash) { + console.log("Found altblock with " + blockData.hash + " hash"); + blockData.pay_value = global.support.decimalToCoin(pay); + blockData.unlocked = false; + console.log("Put " + blockData.pay_value + " pay_value to block"); + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + txn.commit(); + cursor.close(); + console.log("Changed altblock"); + process.exit(0); + } + }); + } + cursor.close(); + txn.commit(); + console.log("Not found altblock with " + hash + " hash"); + process.exit(1); +}); diff --git a/manage_scripts/altblock_recalc_distro.js b/manage_scripts/altblock_recalc_distro.js new file mode 100644 index 00000000..73bc1853 --- /dev/null +++ b/manage_scripts/altblock_recalc_distro.js @@ -0,0 +1,44 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.hash) { + console.error("Please specify altblock hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let is_found = true; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash === hash) { + is_found = true; + global.coinFuncs.getPortBlockHeaderByHash(blockData.port, hash, (err, body) => { + if (err !== null) { + console.log("Altblock with " + hash + " hash still has invalid hash for " + blockData.port + " port! Exiting!"); + cursor.close(); + txn.commit(); + process.exit(1); + } + console.log("Changing alt-block pay_ready from " + blockData.pay_ready + " to false"); + blockData.pay_ready = false; + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + cursor.close(); + txn.commit(); + console.log("Altblock with " + hash + " hash was validated! Exiting!"); + process.exit(0); + }); + } + }); + } + if (!is_found) { + cursor.close(); + txn.commit(); + console.log("Not found altblock with " + hash + " hash"); + process.exit(1); + } +}); diff --git a/manage_scripts/altblock_revalidate.js b/manage_scripts/altblock_revalidate.js new file mode 100644 index 00000000..4bcba363 --- /dev/null +++ b/manage_scripts/altblock_revalidate.js @@ -0,0 +1,53 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.hash) { + console.error("Please specify altblock hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let is_found = true; + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash === hash) { + is_found = true; + global.coinFuncs.getPortBlockHeaderByHash(blockData.port, hash, (err, body) => { + if (err !== null || !body.reward) { + if (blockData.valid) { + blockData.valid = false; + blockData.unlocked = true; + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + console.log("Altblock with " + hash + " hash became invalid for " + blockData.port + " port! Exiting!"); + } else { + console.log("Altblock with " + hash + " hash still has invalid hash for " + blockData.port + " port! Exiting!"); + } + cursor.close(); + txn.commit(); + process.exit(1); + } + blockData.valid = true; + blockData.unlocked = false; + //if (blockData.value != body.reward) console.log("Changing alt-block value from " + blockData.value + " to " + body.reward); + //blockData.value = body.reward; + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + cursor.close(); + txn.commit(); + console.log("Altblock with " + hash + " hash was validated! Exiting!"); + process.exit(0); + }); + } + }); + } + if (!is_found) { + cursor.close(); + txn.commit(); + console.log("Not found altblock with " + hash + " hash"); + process.exit(1); + } +}); diff --git a/manage_scripts/altblock_unlock_without_pay.js b/manage_scripts/altblock_unlock_without_pay.js new file mode 100644 index 00000000..f510982a --- /dev/null +++ b/manage_scripts/altblock_unlock_without_pay.js @@ -0,0 +1,16 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2), { '--': true }); + +let hashes = []; +for (const h of argv['--']) { + hashes.push(h); +} + +require("../init_mini.js").init(function() { + hashes.forEach(function(hash) { + global.database.unlockAltBlock(hash); + console.log("Altblock with " + hash + " hash un-locked!"); + }) + process.exit(0); +}); diff --git a/manage_scripts/altblocks_pay_manually.js b/manage_scripts/altblocks_pay_manually.js new file mode 100644 index 00000000..ebb2b648 --- /dev/null +++ b/manage_scripts/altblocks_pay_manually.js @@ -0,0 +1,39 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2), { '--': true }); + +let hashes = {}; +for (const h of argv['--']) { + hashes[h] = 1; +} + +if (!argv.pay) { + console.error("Please specify pay value in main currency"); + process.exit(1); +} +const pay = argv.pay; + +require("../init_mini.js").init(function() { + let changed = 0; + let txn = global.database.env.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (blockData.hash in hashes) { + console.log("Found altblock with " + blockData.hash + " hash"); + blockData.pay_value = global.support.decimalToCoin(pay); + blockData.unlocked = false; + console.log("Put " + blockData.pay_value + " pay_value to block"); + txn.putBinary(global.database.altblockDB, key, global.protos.AltBlock.encode(blockData)); + console.log("Changed altblock"); + changed += 1; + } + }); + } + cursor.close(); + txn.commit(); + if (!changed) console.log("Not found altblocks with specified hashes"); + else console.log("Changed " + changed + " blocks"); + process.exit(0); +}); \ No newline at end of file diff --git a/manage_scripts/block_add.js b/manage_scripts/block_add.js new file mode 100644 index 00000000..08ae6820 --- /dev/null +++ b/manage_scripts/block_add.js @@ -0,0 +1,51 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.height) { + console.error("Please specify block height"); + process.exit(1); +} +const height = argv.height; + +if (!argv.body) { + console.error("Please specify block body"); + process.exit(1); +} +const body = argv.body; +let body2; + +try { body2 = JSON.parse(body); } catch(e) { + console.error("Can't parse block body: " + body); + process.exit(1); +} + +require("../init_mini.js").init(function() { + const body3 = { + "hash": body2.hash, + "difficulty": body2.difficulty, + "shares": body2.shares, + "timestamp": body2.timestamp, + "poolType": body2.poolType, + "unlocked": body2.unlocked, + "valid": body2.valid, + "value": body2.value + }; + if (typeof (body3.hash) === 'undefined' || + typeof (body3.difficulty) === 'undefined' || + typeof (body3.shares) === 'undefined' || + typeof (body3.timestamp) === 'undefined' || + typeof (body3.poolType) === 'undefined' || + typeof (body3.unlocked) === 'undefined' || + typeof (body3.valid) === 'undefined' || + typeof (body3.value) === 'undefined') { + console.error("Block body is invalid: " + JSON.stringify(body3)); + process.exit(1); + } + const body4 = global.protos.Block.encode(body3); + let txn = global.database.env.beginTxn(); + txn.putBinary(global.database.blockDB, height, body4); + txn.commit(); + console.log("Block on " + height + " height added! Exiting!"); + process.exit(0); +}); diff --git a/manage_scripts/block_lock_to_pay.js b/manage_scripts/block_lock_to_pay.js new file mode 100644 index 00000000..0d48b0c1 --- /dev/null +++ b/manage_scripts/block_lock_to_pay.js @@ -0,0 +1,22 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.height) { + console.error("Please specify block height to lock again (to pay it again)"); + process.exit(1); +} +const height = argv.height; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn(); + let blockProto = txn.getBinary(global.database.blockDB, parseInt(height)); + if (blockProto !== null) { + let blockData = global.protos.Block.decode(blockProto); + blockData.unlocked = false; + txn.putBinary(global.database.blockDB, height, global.protos.Block.encode(blockData)); + } + txn.commit(); + console.log("Block on " + height + " height re-locked! Exiting!"); + process.exit(0); +}); diff --git a/manage_scripts/block_unlock_without_pay.js b/manage_scripts/block_unlock_without_pay.js new file mode 100644 index 00000000..df740371 --- /dev/null +++ b/manage_scripts/block_unlock_without_pay.js @@ -0,0 +1,15 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.hash) { + console.error("Please specify block hash to unlock it (and avoid payment)"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + global.database.unlockBlock(hash); + console.log("Block on " + hash + " height un-locked! Exiting!"); + process.exit(0); +}); diff --git a/manage_scripts/cache_clean.js b/manage_scripts/cache_clean.js new file mode 100644 index 00000000..3dd69f69 --- /dev/null +++ b/manage_scripts/cache_clean.js @@ -0,0 +1,103 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); +const user = argv.user ? argv.user : null; + +let count = 0; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.cacheDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentString(function(key, data){ // jshint ignore:line + if (user && !key.includes(user)) return; + if (key.length < 95) { // min XMR address length + console.log("Skipping " + key + " key"); + return; + } + if (key.includes("identifiers:")) { + let parts = key.split(/:(.+)/); + let key2 = parts[1]; + + try { + let data2 = JSON.parse(data); + if (data2.length == 0) return; + let isAlive = false; + for (let i in data2) { + let stats = global.database.getCache("stats:" + key2 + "_" + data2[i]); + if (stats && Date.now() - stats.lastHash <= 24*60*60*1000) isAlive = true; + } + if (!isAlive) { + data2 = []; + console.log(key + ": found dead key"); + let txn2 = global.database.env.beginTxn(); + txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); + txn2.commit(); + } + + } catch (e) { + console.error("Bad cache data with " + key + " key"); + } + + } else if (key.includes("_")) { + if (key.includes("history:") || key.includes("stats:")) { + let parts = key.split(/:(.+)/); + let key2 = parts[1]; + if (!global.database.getCache(key2)) { + //console.log(key + ": found orphan key"); + } + } else { + let stats = global.database.getCache("stats:" + key); + if (!stats) { + console.log(key + ": found key without stats: " + data); + let txn2 = global.database.env.beginTxn(); + txn2.del(global.database.cacheDB, key); + if (global.database.getCache("history:" + key)) txn2.del(global.database.cacheDB, "history:" + key); + txn2.commit(); + ++ count; + return; + } + if (!global.database.getCache("history:" + key)) { + console.log(key + ": found key without history: " + data); + let txn2 = global.database.env.beginTxn(); + txn2.del(global.database.cacheDB, key); + txn2.del(global.database.cacheDB, "stats:" + key); + txn2.commit(); + ++ count; + return; + } + if (Date.now() - stats.lastHash > 7*24*60*60*1000) { + console.log(key + ": found outdated key"); + let txn2 = global.database.env.beginTxn(); + txn2.del(global.database.cacheDB, key); + txn2.del(global.database.cacheDB, "history:" + key); + txn2.del(global.database.cacheDB, "stats:" + key); + txn2.commit(); + ++ count; + } + + } + + } else if (key.includes("stats:")) { + try { + let data2 = JSON.parse(data); + if ((data2.hash || data2.hash2) && Date.now() - data2.lastHash > 24*60*60*1000) { + console.log(key + ": found dead account"); + data2.hash = data2.hash2 = 0; + let txn2 = global.database.env.beginTxn(); + txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); + txn2.commit(); + } + } catch (e) { + console.error("Bad cache data with " + key + " key"); + } + + } + }); + } + + cursor.close(); + txn.commit(); + console.log("Deleted items: " + count); + process.exit(0); +}); diff --git a/manage_scripts/cache_get.js b/manage_scripts/cache_get.js new file mode 100644 index 00000000..1feb70f8 --- /dev/null +++ b/manage_scripts/cache_get.js @@ -0,0 +1,21 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.key) { + console.error("Please specify key"); + process.exit(1); +} +const key = argv.key; + +require("../init_mini.js").init(function() { + let value = global.database.getCache(key); + if (value !== false) { + console.log(JSON.stringify(value)); + process.exit(0); + } else { + console.error("Key is not found"); + process.exit(1); + } +}); + diff --git a/manage_scripts/cache_set.js b/manage_scripts/cache_set.js new file mode 100644 index 00000000..e1f1da05 --- /dev/null +++ b/manage_scripts/cache_set.js @@ -0,0 +1,27 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.key) { + console.error("Please specify key"); + process.exit(1); +} +const key = argv.key; + +if (!argv.value) { + console.error("Please specify value"); + process.exit(1); +} +const value = argv.value; + +require("../init_mini.js").init(function() { + try { + let value2 = JSON.parse(value); + global.database.setCache(key, value2); + process.exit(0); + } catch(e) { + console.error("Can't parse your value: " + value); + process.exit(1); + } +}); + diff --git a/manage_scripts/cache_upgrade.js b/manage_scripts/cache_upgrade.js new file mode 100644 index 00000000..c2569c65 --- /dev/null +++ b/manage_scripts/cache_upgrade.js @@ -0,0 +1,78 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); +const user = argv.user ? argv.user : null; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.cacheDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentString(function(key, data){ // jshint ignore:line + if (key.length < 95) { // min XMR address length + if (key.includes("history:") || key.includes("stats:") || key.includes("identifiers:")) { + console.log(key + ": removing bad key"); + let txn2 = global.database.env.beginTxn(); + txn2.del(global.database.cacheDB, key); + txn2.commit(); + return; + } + console.log("Skipping " + key + " key"); + return; + } + if (key.includes("history:") || key.includes("stats:") || key.includes("identifiers:")) return; + if (user && !key.includes(user)) return; + + let txn2 = global.database.env.beginTxn(); + if (key.includes("_identifiers")) { + let parts = key.split("_"); + let key2 = parts[0]; + if (global.database.getCache("identifiers:" + key2)) { + console.log(key2 + ": removing outdated _identifiers"); + txn2.del(global.database.cacheDB, key); + } else { + console.log(key2 + ": moving _identifiers to identifiers:"); + txn2.putString(global.database.cacheDB, "identifiers:" + key2, data); + txn2.del(global.database.cacheDB, key); + } + } else { + try { + let data2 = JSON.parse(data); + if ("hash" in data2 && "lastHash" in data2) { + if (global.database.getCache("stats:" + key)) { + console.log(key + ": removing outdated stats"); + delete data2["hash"]; + delete data2["lastHash"]; + txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); + } else { + console.log(key + ": moving old stats to stats:"); + let data3 = { hash: data2.hash, lastHash: data2.lastHash }; + delete data2["hash"]; + delete data2["lastHash"]; + txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); + txn2.putString(global.database.cacheDB, "stats:" + key, JSON.stringify(data3)); + } + } + if ("hashHistory" in data2) { + if (global.database.getCache("history:" + key)) { + console.log(key + ": removing outdated history"); + delete data2["hashHistory"]; + txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); + } else { + console.log(key + ": moving old history to history:"); + let data3 = { hashHistory: data2.hashHistory }; + delete data2["hashHistory"]; + txn2.putString(global.database.cacheDB, key, JSON.stringify(data2)); + txn2.putString(global.database.cacheDB, "history:" + key, JSON.stringify(data3)); + } + } + } catch (e) { + console.error("Bad cache data with " + key + " key"); + } + } + txn2.commit(); + }); + } + cursor.close(); + txn.commit(); + process.exit(0); +}); diff --git a/manage_scripts/clean_altblocks.js b/manage_scripts/clean_altblocks.js new file mode 100644 index 00000000..21d36f5e --- /dev/null +++ b/manage_scripts/clean_altblocks.js @@ -0,0 +1,38 @@ +"use strict"; + +require("../init_mini.js").init(function() { + console.log("Cleaning up the alt block DB. Searching for items to delete"); + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + let deleted = []; + let block_count = {}; + for (let found = cursor.goToLast(); found; found = cursor.goToPrev()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + if (!(blockData.port in block_count)) block_count[blockData.port] = 0; + ++ block_count[blockData.port]; + if (blockData.unlocked && (block_count[blockData.port] > 20000 || Date.now() - blockData.timestamp > 3*365*24*60*60*1000)) { + deleted.push(key); + } + }); + } + + cursor.close(); + txn.commit(); + + console.log("Deleting altblock items: " + deleted.length); + + let chunkSize = 0; + txn = global.database.env.beginTxn(); + deleted.forEach(function(key) { + ++ chunkSize; + txn.del(global.database.altblockDB, key); + if (chunkSize > 500) { + txn.commit(); + txn = global.database.env.beginTxn(); + chunkSize = 0; + } + }); + txn.commit(); + process.exit(0); +}); diff --git a/manage_scripts/dump_altblocks.js b/manage_scripts/dump_altblocks.js new file mode 100644 index 00000000..9b42df24 --- /dev/null +++ b/manage_scripts/dump_altblocks.js @@ -0,0 +1,16 @@ +"use strict"; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn({readOnly: true}); + + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.AltBlock.decode(data); + console.log(key + ": " + JSON.stringify(blockData)) + }); + } + cursor.close(); + txn.commit(); + process.exit(0); +}); diff --git a/manage_scripts/dump_blocks.js b/manage_scripts/dump_blocks.js new file mode 100644 index 00000000..8f589712 --- /dev/null +++ b/manage_scripts/dump_blocks.js @@ -0,0 +1,16 @@ +"use strict"; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn({readOnly: true}); + + let cursor = new global.database.lmdb.Cursor(txn, global.database.blockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let blockData = global.protos.Block.decode(data); + console.log(key + ": " + JSON.stringify(blockData)) + }); + } + cursor.close(); + txn.commit(); + process.exit(0); +}); diff --git a/manage_scripts/dump_cache.js b/manage_scripts/dump_cache.js new file mode 100644 index 00000000..eb2d2603 --- /dev/null +++ b/manage_scripts/dump_cache.js @@ -0,0 +1,17 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); +const user = argv.user ? argv.user : null; + +require("../init_mini.js").init(function() { + let txn = global.database.env.beginTxn({readOnly: true}); + let cursor = new global.database.lmdb.Cursor(txn, global.database.cacheDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentString(function(key, data){ // jshint ignore:line + if (!user || key.includes(user)) console.log(key + ": " + data); + }); + } + cursor.close(); + txn.commit(); + process.exit(0); +}); diff --git a/manage_scripts/dump_shares.js b/manage_scripts/dump_shares.js new file mode 100644 index 00000000..401371d0 --- /dev/null +++ b/manage_scripts/dump_shares.js @@ -0,0 +1,51 @@ +"use strict"; + +let range = require('range'); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to dump"); + process.exit(1); +} +const user = argv.user; + +let paymentid; +if (argv.paymentid) paymentid = argv.paymentid; + +let worker; +if (argv.worker) worker = argv.worker; + +let depth = 10; +if (argv.depth) depth = argv.depth; + +console.log("Dumping shares for " + user + " user"); +if (paymentid) console.log("Dumping shares for " + paymentid + " paymentid"); +if (worker) console.log("Dumping shares for " + worker + " worker"); + +require("../init_mini.js").init(function() { + + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err !== null) { + console.error("Invalid block header"); + process.exit(1); + } + let lastBlock = body.height + 1; + let txn = global.database.env.beginTxn({readOnly: true}); + + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + range.range(lastBlock, lastBlock - depth, -1).forEach(function (blockID) { + for (let found = (cursor.goToRange(parseInt(blockID)) === blockID); found; found = cursor.goToNextDup()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let shareData = global.protos.Share.decode(data); + if (shareData.paymentAddress === user && (!paymentid || shareData.paymentID === paymentid) && (!worker || shareData.identifier === worker)) { + var d = new Date(shareData.timestamp); + console.log(d.toString() + ": " + JSON.stringify(shareData)) + } + }); + } + }); + cursor.close(); + txn.commit(); + process.exit(0); + }); +}); diff --git a/manage_scripts/dump_shares_all.js b/manage_scripts/dump_shares_all.js new file mode 100644 index 00000000..7846e21b --- /dev/null +++ b/manage_scripts/dump_shares_all.js @@ -0,0 +1,35 @@ +"use strict"; + +let range = require('range'); +const argv = require('minimist')(process.argv.slice(2)); + +let depth = 10; +if (argv.depth) depth = argv.depth; + +console.log("Dumping shares"); + +require("../init_mini.js").init(function() { + + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err !== null) { + console.error("Invalid block header"); + process.exit(1); + } + let lastBlock = body.height + 1; + let txn = global.database.env.beginTxn({readOnly: true}); + + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + range.range(lastBlock, lastBlock - depth, -1).forEach(function (blockID) { + for (let found = (cursor.goToRange(parseInt(blockID)) === blockID); found; found = cursor.goToNextDup()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let shareData = global.protos.Share.decode(data); + var d = new Date(shareData.timestamp); + console.log(d.toString() + ": " + JSON.stringify(shareData)) + }); + } + }); + cursor.close(); + txn.commit(); + process.exit(0); + }); +}); diff --git a/manage_scripts/dump_shares_port.js b/manage_scripts/dump_shares_port.js new file mode 100644 index 00000000..485de4e6 --- /dev/null +++ b/manage_scripts/dump_shares_port.js @@ -0,0 +1,43 @@ +"use strict"; + +let range = require('range'); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port to dump"); + process.exit(1); +} +const port = argv.port; + +let depth = 10; +if (argv.depth) depth = argv.depth; + +console.log("Dumping shares for " + port + " port"); + +require("../init_mini.js").init(function() { + + global.coinFuncs.getLastBlockHeader(function (err, body) { + if (err !== null) { + console.error("Invalid block header"); + process.exit(1); + } + let lastBlock = body.height + 1; + let txn = global.database.env.beginTxn({readOnly: true}); + + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + range.range(lastBlock, lastBlock - depth, -1).forEach(function (blockID) { + for (let found = (cursor.goToRange(parseInt(blockID)) === blockID); found; found = cursor.goToNextDup()) { + cursor.getCurrentBinary(function(key, data){ // jshint ignore:line + let shareData = global.protos.Share.decode(data); + if (shareData.port === port) { + var d = new Date(shareData.timestamp); + console.log(d.toString() + ": " + JSON.stringify(shareData)) + } + }); + } + }); + cursor.close(); + txn.commit(); + process.exit(0); + }); +}); diff --git a/manage_scripts/fix_negative_ex_xmr_balance.js b/manage_scripts/fix_negative_ex_xmr_balance.js new file mode 100644 index 00000000..072acf42 --- /dev/null +++ b/manage_scripts/fix_negative_ex_xmr_balance.js @@ -0,0 +1,25 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +require("../init_mini.js").init(function() { + const xmr_balance = global.database.getCache("xmr_balance"); + if (xmr_balance !== false) { + if (!xmr_balance.value || xmr_balance.value < 0) { + console.error("Can't fix xmr_balance: " + JSON.stringify(xmr_balance)); + process.exit(1); + return; + } + const xmr_balance2 = { value: -xmr_balance.expected_increase, expected_increase: xmr_balance.expected_increase }; + console.log("In 10 seconds is going to change xmr_balance from " + JSON.stringify(xmr_balance) + " into " + JSON.stringify(xmr_balance2)); + setTimeout(function() { + global.database.setCache("xmr_balance", xmr_balance2); + console.log("Done."); + process.exit(0); + }, 10*1000); + } else { + console.error("Key xmr_balance is not found"); + process.exit(1); + } +}); + diff --git a/manage_scripts/get_block_hash.js b/manage_scripts/get_block_hash.js new file mode 100644 index 00000000..79240549 --- /dev/null +++ b/manage_scripts/get_block_hash.js @@ -0,0 +1,23 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +if (!argv.hash) { + console.error("Please specify hash"); + process.exit(1); +} +const hash = argv.hash; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortAnyBlockHeaderByHash(port, hash, false, function (err_header, body_header) { + console.log("err:" + JSON.stringify(err_header)); + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/get_block_header.js b/manage_scripts/get_block_header.js new file mode 100644 index 00000000..4f576074 --- /dev/null +++ b/manage_scripts/get_block_header.js @@ -0,0 +1,17 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortLastBlockHeader(port, function (err_header, body_header) { + console.log("err:" + JSON.stringify(err_header)); + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/get_block_height.js b/manage_scripts/get_block_height.js new file mode 100644 index 00000000..297877d2 --- /dev/null +++ b/manage_scripts/get_block_height.js @@ -0,0 +1,23 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +if (!argv.height) { + console.error("Please specify height"); + process.exit(1); +} +const height = argv.height; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortBlockHeaderByID(port, height, function (err_header, body_header) { + console.log("err:" + JSON.stringify(err_header)); + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/get_block_template.js b/manage_scripts/get_block_template.js new file mode 100644 index 00000000..9e5e4d8d --- /dev/null +++ b/manage_scripts/get_block_template.js @@ -0,0 +1,16 @@ +"use strict"; + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.port) { + console.error("Please specify port"); + process.exit(1); +} +const port = argv.port; + +require("../init_mini.js").init(function() { + global.coinFuncs.getPortBlockTemplate(port, function (body_header) { + console.log("body:" + JSON.stringify(body_header)); + process.exit(0); + }); +}); diff --git a/manage_scripts/mdb_copy.js b/manage_scripts/mdb_copy.js new file mode 100644 index 00000000..64c12d3b --- /dev/null +++ b/manage_scripts/mdb_copy.js @@ -0,0 +1,120 @@ +"use strict"; + +const lmdb = require('node-lmdb'); +const fs = require('fs'); + +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.dir) { + console.error("Please specify output lmdb dir"); + process.exit(1); +} + +if (fs.existsSync(argv.dir + "/data.mdb")) { + console.error("Please specify empty output lmdb dir"); + process.exit(1); +} + +if (!argv.size) { + console.error("Please specify output lmdb size in GB"); + process.exit(1); +} + +require("../init_mini.js").init(function() { + let env2 = new lmdb.Env(); + env2.open({ + path: argv.dir, + maxDbs: 10, + mapSize: argv.size * 1024 * 1024 * 1024, + useWritemap: true, + maxReaders: 512 + }); + let shareDB2 = env2.openDbi({ + name: 'shares', + create: true, + dupSort: true, + dupFixed: false, + integerDup: true, + integerKey: true, + keyIsUint32: true + }); + let blockDB2 = env2.openDbi({ + name: 'blocks', + create: true, + integerKey: true, + keyIsUint32: true + }); + let altblockDB2 = env2.openDbi({ + name: 'altblocks', + create: true, + integerKey: true, + keyIsUint32: true + }); + let cacheDB2 = env2.openDbi({ + name: 'cache', + create: true + }); + + console.log("Copying blocks"); + { + let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.blockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(blockDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + console.log("Copying altblocks"); + { let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.altblockDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(altblockDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + console.log("Copying shares"); + { let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.shareDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(shareDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + console.log("Copying cache"); + { let txn = global.database.env.beginTxn({readOnly: true}); + let txn2 = env2.beginTxn(); + let cursor = new global.database.lmdb.Cursor(txn, global.database.cacheDB); + for (let found = cursor.goToFirst(); found; found = cursor.goToNext()) { + cursor.getCurrentBinary(function(key, data) { + txn2.putBinary(cacheDB2, key, data); + }); + } + cursor.close(); + txn.commit(); + txn2.commit(); + } + + + + env2.close(); + console.log("DONE"); + process.exit(0); +}); diff --git a/manage_scripts/news.sh b/manage_scripts/news.sh new file mode 100755 index 00000000..c1fe4479 --- /dev/null +++ b/manage_scripts/news.sh @@ -0,0 +1,9 @@ +#!/bin/bash + +subject=$1 +body=$2 + +if [ -z "$subject" ]; then echo "Set subject as first script paaameter"; exit 1; fi +if [ -z "$body" ]; then echo "Set bosy as second script paaameter"; exit 1; fi + +node cache_set.js --key=news --value='{"created": "'$(date +%s)'", "subject": "'"$subject"'", "body": "'"$body"'"}' \ No newline at end of file diff --git a/manage_scripts/user_del.js b/manage_scripts/user_del.js new file mode 100644 index 00000000..19b8b57a --- /dev/null +++ b/manage_scripts/user_del.js @@ -0,0 +1,120 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to delete"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + const parts = user.split("."); + const address = parts.length === 1 ? user : parts[0]; + const payment_id = parts.length === 2 ? parts[1] : null; + + console.log("Address: " + address); + console.log("PaymentID: " + payment_id); + console.log("Max payment to remove: " + global.config.payout.walletMin); + let rows2remove = 0; + + const where_str = payment_id === null ? "payment_address = '" + address + "' AND (payment_id IS NULL OR payment_id = '')" + : "payment_address = '" + address + "' AND payment_id = '" + payment_id + "'"; + + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length > 1) { + console.error("Too many users were selected!"); + process.exit(1); + } + console.log("Found rows in users table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + where_str).then(function (rows) { + if (rows.length > 1) { + console.error("Too many users were selected!"); + process.exit(1); + } + if (rows.length === 1 && rows[0].amount >= global.support.decimalToCoin(global.config.payout.walletMin)) { + console.error("Too big payment left: " + global.support.coinToDecimal(rows[0].amount)); + process.exit(1); + } + if (rows.length) { + console.log("Balance last update time: " + rows[0].last_edited); + if (Date.now()/1000 - global.support.formatDateFromSQL(rows[0].last_edited) < 12*60*60) { + console.error("There was recent amount update. Refusing to continue!"); + process.exit(1); + } + } + console.log("Found rows in balance table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM payments WHERE " + where_str).then(function (rows) { + console.log("Found rows in payments table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + const address = global.database.getCache(user); + const stats = global.database.getCache("stats:" + user); + const history = global.database.getCache("history:" + user); + const identifiers = global.database.getCache("identifiers:" + user); + + if (address != false) console.log("Cache key is not empty: " + user); + if (stats != false) console.log("Cache key is not empty: " + "stats:" + user); + if (history != false) console.log("Cache key is not empty: " + "history:" + user); + if (identifiers != false) console.log("Cache key is not empty: " + "identifiers:" + user); + callback(); + + }, + function (callback) { + if (!rows2remove) { // to check that we accidently do not remove something usefull from LMDB cache + console.error("User was not found in SQL. Refusing to proceed to LMDB cache cleaning"); + process.exit(1); + } + callback(); + + }, + function (callback) { + global.mysql.query("DELETE FROM users WHERE username = ?", [user]).then(function (rows) { + console.log("DELETE FROM users WHERE username = " + user); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM balance WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM balance WHERE " + where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM payments WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM payments WHERE " + where_str); + callback(); + }); + }, + function (callback) { + console.log("Deleting LMDB cache keys"); + let txn = global.database.env.beginTxn(); + if (global.database.getCache(user)) txn.del(global.database.cacheDB, user); + if (global.database.getCache("stats:" + user)) txn.del(global.database.cacheDB, "stats:" + user); + if (global.database.getCache("history:" + user)) txn.del(global.database.cacheDB, "history:" + user); + if (global.database.getCache("identifiers:" + user)) txn.del(global.database.cacheDB, "identifiers:" + user); + txn.commit(); + callback(); + }, + function (callback) { + console.log("DONE"); + process.exit(0); + } + ]); +}); diff --git a/manage_scripts/user_del_force.js b/manage_scripts/user_del_force.js new file mode 100644 index 00000000..18cfe3a7 --- /dev/null +++ b/manage_scripts/user_del_force.js @@ -0,0 +1,126 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to delete"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + const parts = user.split("."); + const address = parts.length === 1 ? user : parts[0]; + const payment_id = parts.length === 2 ? parts[1] : null; + + console.log("Address: " + address); + console.log("PaymentID: " + payment_id); + console.log("Max payment to remove: " + global.config.payout.walletMin); + let rows2remove = 0; + + const where_str = payment_id === null ? "payment_address = '" + address + "' AND (payment_id IS NULL OR payment_id = '')" + : "payment_address = '" + address + "' AND payment_id = '" + payment_id + "'"; + + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length > 1) { + console.error("Too many users were selected!"); + process.exit(1); + } + console.log("Found rows in users table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + where_str).then(function (rows) { + if (rows.length > 1) { + console.error("Too many users were selected!"); + process.exit(1); + } + if (rows.length === 1 && rows[0].amount >= global.support.decimalToCoin(global.config.payout.walletMin)) { + console.error("Too big payment left: " + global.support.coinToDecimal(rows[0].amount)); + process.exit(1); + } + console.log("Found rows in balance table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM payments WHERE " + where_str).then(function (rows) { + console.log("Found rows in payments table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM block_balance WHERE " + where_str).then(function (rows) { + console.log("Found rows in block_balance table: " + rows.length); + rows2remove += rows.length; + callback(); + }); + }, + function (callback) { + const address = global.database.getCache(user); + const stats = global.database.getCache("stats:" + user); + const history = global.database.getCache("history:" + user); + const identifiers = global.database.getCache("identifiers:" + user); + + if (address != false) console.log("Cache key is not empty: " + user); + if (stats != false) console.log("Cache key is not empty: " + "stats:" + user); + if (history != false) console.log("Cache key is not empty: " + "history:" + user); + if (identifiers != false) console.log("Cache key is not empty: " + "identifiers:" + user); + callback(); + + }, + function (callback) { + if (!rows2remove) { // to check that we accidently do not remove something usefull from LMDB cache + console.error("User was not found in SQL. Refusing to proceed to LMDB cache cleaning"); + process.exit(1); + } + callback(); + + }, + function (callback) { + global.mysql.query("DELETE FROM users WHERE username = ?", [user]).then(function (rows) { + console.log("DELETE FROM users WHERE username = " + user); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM balance WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM balance WHERE " + where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM payments WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM payments WHERE " + where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM block_balance WHERE " + where_str).then(function (rows) { + console.log("DELETE FROM block_balance WHERE " + where_str); + callback(); + }); + }, + function (callback) { + console.log("Deleting LMDB cache keys"); + let txn = global.database.env.beginTxn(); + if (global.database.getCache(user)) txn.del(global.database.cacheDB, user); + if (global.database.getCache("stats:" + user)) txn.del(global.database.cacheDB, "stats:" + user); + if (global.database.getCache("history:" + user)) txn.del(global.database.cacheDB, "history:" + user); + if (global.database.getCache("identifiers:" + user)) txn.del(global.database.cacheDB, "identifiers:" + user); + txn.commit(); + callback(); + }, + function (callback) { + console.log("DONE"); + process.exit(0); + } + ]); +}); diff --git a/package.json b/package.json index fb39ed91..a826220b 100644 --- a/package.json +++ b/package.json @@ -1,48 +1,40 @@ { "name": "nodejs-pool", "version": "0.0.1", - "description": "Fairly simple universal cryptonote pool", + "description": "Improved version of Snipa22 nodejs-pool", "main": "init.js", "repository": { "type": "git", - "url": "https://github.com/Snipa22/node-crypto-pool.git" + "url": "https://github.com/MoneroOcean/nodejs-pool.git" }, - "author": "Alexander Blair", + "author": "Multiple", "license": "MIT", "dependencies": { - "async": "2.1.4", - "bignum": "^0.12.5", - "bluebird": "3.4.7", + "apicache": "1.2.1", + "async": "3.2.2", + "bignum": "^0.13.1", "body-parser": "^1.16.0", - "bufferutil": "^1.3.0", "circular-buffer": "1.0.2", "cluster": "0.7.7", "concat-stream": "^1.6.0", "cors": "^2.8.1", - "crypto": "0.0.3", - "debug": "2.5.1", - "express": "4.14.0", - "jsonwebtoken": "^7.2.1", - "minimist": "1.2.0", - "moment": "2.17.1", - "mysql": "2.12.0", - "node-lmdb": "0.4.12", + "crypto-js": "^4.2.0", + "cryptoforknote-util": "git+https://github.com/MoneroOcean/node-cryptoforknote-util.git#v15.6.2", + "cryptonight-hashing": "git+https://github.com/MoneroOcean/node-cryptonight-hashing.git#v29.0.6", + "debug": "2.6.9", + "express": "^4.17.1", + "jsonwebtoken": "^9.0.2", + "minimist": ">=1.2.6", + "moment": "2.29.4", + "mysql": "2.18.1", + "node-lmdb": "git+https://github.com/Venemo/node-lmdb.git#c3135a3809da1d64ce1f0956b37b618711e33519", "promise-mysql": "3.0.0", - "protocol-buffers": "^3.2.1", + "protocol-buffers": "5.0.0", "range": "0.0.3", - "redis": "^2.6.5", "request": "^2.79.0", - "request-json": "0.6.1", - "shapeshift.io": "1.3.0", - "socketio": "^1.0.0", + "request-json": "^0.1.0", "sprintf-js": "^1.0.3", - "sticky-cluster": "^0.3.1", - "uuid": "3.0.1", - "wallet-address-validator": "0.1.0", - "zmq": "^2.15.3" - }, - "optionalDependencies": { - "cryptonote-util": "git://github.com/Snipa22/node-cryptonote-util.git#xmr-Nan-2.0", - "multi-hashing": "git+https://github.com/Snipa22/node-multi-hashing-aesni.git" + "utf8": "^3.0.0", + "wallet-address-validator": "0.1.0" } } diff --git a/sample_config.sql b/sample_config.sql deleted file mode 100644 index dac267ec..00000000 --- a/sample_config.sql +++ /dev/null @@ -1,6 +0,0 @@ -UPDATE pool.config SET item_value = '' WHERE module = 'pool' and item = 'address'; -UPDATE pool.config SET item_value = '' WHERE module = 'payout' and item = 'feeAddress'; -UPDATE pool.config SET item_value = '' WHERE module = 'general' and item = 'mailgunKey'; -UPDATE pool.config SET item_value = '' WHERE module = 'general' and item = 'mailgunURL'; -UPDATE pool.config SET item_value = '' WHERE module = 'general' and item = 'emailFrom'; -UPDATE pool.config SET item_value = '' WHERE module = 'general' and item = 'shareHost'; diff --git a/sql_sync/config_entries.json b/sql_sync/config_entries.json deleted file mode 100644 index 8d320da0..00000000 --- a/sql_sync/config_entries.json +++ /dev/null @@ -1,578 +0,0 @@ -[ - { - "id": 1, - "module": "pool", - "item": "address", - "item_value": "", - "item_type": "string", - "Item_desc": "Address to mine to, this should be the wallet-rpc address." - }, - { - "id": 2, - "module": "pool", - "item": "minerTimeout", - "item_value": "900", - "item_type": "int", - "Item_desc": "Length of time before a miner is flagged inactive." - }, - { - "id": 3, - "module": "pool", - "item": "banEnabled", - "item_value": "true", - "item_type": "bool", - "Item_desc": "Enables/disabled banning of \"bad\" miners." - }, - { - "id": 4, - "module": "pool", - "item": "banLength", - "item_value": "-15m", - "item_type": "string", - "Item_desc": "Ban duration except perma-bans" - }, - { - "id": 5, - "module": "pool", - "item": "targetTime", - "item_value": "30", - "item_type": "int", - "Item_desc": "Time in seconds between share finds" - }, - { - "id": 6, - "module": "pool", - "item": "trustThreshold", - "item_value": "30", - "item_type": "int", - "Item_desc": "Number of shares before miner trust can kick in." - }, - { - "id": 8, - "module": "pool", - "item": "banPercent", - "item_value": "25", - "item_type": "int", - "Item_desc": "Percentage of shares that need to be invalid to be banned." - }, - { - "id": 9, - "module": "pool", - "item": "banThreshold", - "item_value": "30", - "item_type": "int", - "Item_desc": "Number of shares before bans can begin" - }, - { - "id": 10, - "module": "pool", - "item": "trustedMiners", - "item_value": "true", - "item_type": "bool", - "Item_desc": "Enable the miner trust system" - }, - { - "id": 11, - "module": "pool", - "item": "trustChange", - "item_value": "1", - "item_type": "int", - "Item_desc": "Change in the miner trust in percent" - }, - { - "id": 12, - "module": "pool", - "item": "trustMin", - "item_value": "20", - "item_type": "int", - "Item_desc": "Minimum level of miner trust" - }, - { - "id": 13, - "module": "pool", - "item": "trustPenalty", - "item_value": "30", - "item_type": "int", - "Item_desc": "Number of shares that must be successful to be trusted, reset to this value if trust share is broken" - }, - { - "id": 14, - "module": "pool", - "item": "retargetTime", - "item_value": "60", - "item_type": "int", - "Item_desc": "Time between difficulty retargets" - }, - { - "id": 15, - "module": "daemon", - "item": "address", - "item_value": "127.0.0.1", - "item_type": "string", - "Item_desc": "Monero Daemon RPC IP" - }, - { - "id": 16, - "module": "daemon", - "item": "port", - "item_value": "18081", - "item_type": "int", - "Item_desc": "Monero Daemon RPC Port" - }, - { - "id": 17, - "module": "wallet", - "item": "address", - "item_value": "127.0.0.1", - "item_type": "string", - "Item_desc": "Monero Daemon RPC Wallet IP" - }, - { - "id": 18, - "module": "wallet", - "item": "port", - "item_value": "37458", - "item_type": "int", - "Item_desc": "Monero Daemon RPC Wallet Port" - }, - { - "id": 21, - "module": "rpc", - "item": "https", - "item_value": "false", - "item_type": "bool", - "Item_desc": "Enable RPC over SSL" - }, - { - "id": 22, - "module": "pool", - "item": "maxDifficulty", - "item_value": "500000", - "item_type": "int", - "Item_desc": "Maximum difficulty for VarDiff" - }, - { - "id": 23, - "module": "pool", - "item": "minDifficulty", - "item_value": "100", - "item_type": "int", - "Item_desc": "Minimum difficulty for VarDiff" - }, - { - "id": 24, - "module": "pool", - "item": "varDiffVariance", - "item_value": "20", - "item_type": "int", - "Item_desc": "Percentage out of the target time that difficulty changes" - }, - { - "id": 25, - "module": "pool", - "item": "varDiffMaxChange", - "item_value": "30", - "item_type": "int", - "Item_desc": "Percentage amount that the difficulty may change" - }, - { - "id": 27, - "module": "payout", - "item": "btcFee", - "item_value": "1.5", - "item_type": "float", - "Item_desc": "Fee charged for auto withdrawl via BTC" - }, - { - "id": 28, - "module": "payout", - "item": "ppsFee", - "item_value": "6.5", - "item_type": "float", - "Item_desc": "Fee charged for usage of the PPS pool" - }, - { - "id": 29, - "module": "payout", - "item": "pplnsFee", - "item_value": ".6", - "item_type": "float", - "Item_desc": "Fee charged for the usage of the PPLNS pool" - }, - { - "id": 30, - "module": "payout", - "item": "propFee", - "item_value": ".7", - "item_type": "float", - "Item_desc": "Fee charged for the usage of the proportial pool" - }, - { - "id": 31, - "module": "payout", - "item": "soloFee", - "item_value": ".4", - "item_type": "float", - "Item_desc": "Fee charged for usage of the solo mining pool" - }, - { - "id": 32, - "module": "payout", - "item": "exchangeMin", - "item_value": "5", - "item_type": "float", - "Item_desc": "Minimum XMR balance for payout to exchange/payment ID" - }, - { - "id": 33, - "module": "payout", - "item": "walletMin", - "item_value": ".3", - "item_type": "float", - "Item_desc": "Minimum XMR balance for payout to personal wallet" - }, - { - "id": 34, - "module": "payout", - "item": "devDonation", - "item_value": "5", - "item_type": "float", - "Item_desc": "Donation to XMR core development" - }, - { - "id": 35, - "module": "payout", - "item": "poolDevDonation", - "item_value": "0", - "item_type": "float", - "Item_desc": "Donation to pool developer" - }, - { - "id": 36, - "module": "payout", - "item": "denom", - "item_value": ".000001", - "item_type": "float", - "Item_desc": "Minimum balance that will be paid out to." - }, - { - "id": 37, - "module": "payout", - "item": "blocksRequired", - "item_value": "60", - "item_type": "int", - "Item_desc": "Blocks required to validate a payout before it's performed." - }, - { - "id": 38, - "module": "general", - "item": "sigDivisor", - "item_value": "1000000000000", - "item_type": "int", - "Item_desc": "Divisor for turning coin into human readable amounts " - }, - { - "id": 39, - "module": "payout", - "item": "feeAddress", - "item_value": "", - "item_type": "string", - "Item_desc": "Address that pool fees are sent to." - }, - { - "id": 40, - "module": "payout", - "item": "feesForTXN", - "item_value": "10", - "item_type": "int", - "Item_desc": "Amount of XMR that is left from the fees to pay miner fees." - }, - { - "id": 41, - "module": "payout", - "item": "maxTxnValue", - "item_value": "250", - "item_type": "int", - "Item_desc": "Maximum amount of XMR to send in a single transaction" - }, - { - "id": 42, - "module": "payout", - "item": "shapeshiftPair", - "item_value": "xmr_btc", - "item_type": "string", - "Item_desc": "Pair to use in all shapeshift lookups for auto BTC payout" - }, - { - "id": 43, - "module": "general", - "item": "coinCode", - "item_value": "XMR", - "item_type": "string", - "Item_desc": "Coincode to be loaded up w/ the shapeshift getcoins argument." - }, - { - "id": 44, - "module": "general", - "item": "allowBitcoin", - "item_value": "true", - "item_type": "bool", - "Item_desc": "Allow the pool to auto-payout to BTC via ShapeShift" - }, - { - "id": 45, - "module": "payout", - "item": "exchangeRate", - "item_value": "1217200", - "item_type": "float", - "Item_desc": "Current exchange rate" - }, - { - "id": 46, - "module": "payout", - "item": "bestExchange", - "item_value": "xmrto", - "item_type": "string", - "Item_desc": "Current best exchange" - }, - { - "id": 47, - "module": "payout", - "item": "mixIn", - "item_value": "4", - "item_type": "int", - "Item_desc": "Mixin count for coins that support such things." - }, - { - "id": 48, - "module": "pool", - "item": "geoDNS", - "item_value": "", - "item_type": "string", - "Item_desc": "geoDNS enabled address for the pool." - }, - { - "id": 49, - "module": "general", - "item": "statsBufferLength", - "item_value": "120", - "item_type": "int", - "Item_desc": "Number of items to be cached in the stats buffers." - }, - { - "id": 50, - "module": "api", - "item": "authKey", - "item_value": "", - "item_type": "string", - "Item_desc": "Auth key sent with all Websocket frames for validation" - }, - { - "id": 51, - "module": "general", - "item": "mailgunKey", - "item_value": "", - "item_type": "string", - "Item_desc": "MailGun API Key for notification" - }, - { - "id": 52, - "module": "general", - "item": "mailgunURL", - "item_value": "", - "item_type": "string", - "Item_desc": "MailGun URL for notifications" - }, - { - "id": 53, - "module": "general", - "item": "emailFrom", - "item_value": "", - "item_type": "string", - "Item_desc": "From address for the notification emails" - }, - { - "id": 54, - "module": "pps", - "item": "enable", - "item_value": "false", - "item_type": "bool", - "Item_desc": "Enable PPS or not" - }, - { - "id": 55, - "module": "pplns", - "item": "shareMulti", - "item_value": "2", - "item_type": "int", - "Item_desc": "Multiply this times difficulty to set the N in PPLNS" - }, - { - "id": 56, - "module": "pplns", - "item": "shareMultiLog", - "item_value": "3", - "item_type": "int", - "Item_desc": "How many times the difficulty of the current block do we keep in shares before clearing them out" - }, - { - "id": 57, - "module": "general", - "item": "blockCleaner", - "item_value": "true", - "item_type": "bool", - "Item_desc": "Enable the deletion of blocks or not." - }, - { - "id": 58, - "module": "api", - "item": "secKey", - "item_value": "", - "item_type": "string", - "Item_desc": "HMAC key for Passwords. JWT Secret Key" - }, - { - "id": 59, - "module": "payout", - "item": "feeSlewAmount", - "item_value": ".011", - "item_type": "float", - "Item_desc": "Amount to charge for the txn fee" - }, - { - "id": 60, - "module": "payout", - "item": "feeSlewEnd", - "item_value": "4", - "item_type": "float", - "Item_desc": "Value at which txn fee amount drops to 0" - }, - { - "id": 61, - "module": "general", - "item": "testnet", - "item_value": "false", - "item_type": "bool", - "Item_desc": "Does this pool use testnet?" - }, - { - "id": 62, - "module": "pplns", - "item": "enable", - "item_value": "true", - "item_type": "bool", - "Item_desc": "Enable PPLNS on the pool." - }, - { - "id": 63, - "module": "solo", - "item": "enable", - "item_value": "true", - "item_type": "bool", - "Item_desc": "Enable SOLO mining on the pool" - }, - { - "id": 64, - "module": "general", - "item": "adminEmail", - "item_value": "", - "item_type": "string", - "Item_desc": "Admin e-mail to send e-mails to when something isn't working right." - }, - { - "id": 65, - "module": "payout", - "item": "rpcPasswordEnabled", - "item_value": "false", - "item_type": "bool", - "Item_desc": "Does the wallet use a RPC password?" - }, - { - "id": 66, - "module": "payout", - "item": "rpcPasswordPath", - "item_value": "", - "item_type": "string", - "Item_desc": "Path and file for the RPC password file location" - }, - { - "id": 67, - "module": "payout", - "item": "maxPaymentTxns", - "item_value": "5", - "item_type": "int", - "Item_desc": "Maximum number of transactions in a single payment" - }, - { - "id": 68, - "module": "general", - "item": "shareHost", - "item_value": "", - "item_type": "string", - "Item_desc": "Host that receives share information" - }, - { - "id": 70, - "module": "email", - "item": "workerNotHashingBody", - "item_value": "Hello,\n\nYour worker: %(worker)s has stopped submitting hashes at: %(timestamp)s UTC\n\nThank you,\n%(poolEmailSig)s", - "item_type": "string", - "Item_desc": "Email sent to the miner when their worker stops hashing" - }, - { - "id": 71, - "module": "email", - "item": "workerNotHashingSubject", - "item_value": "Worker %(worker)s stopped hashing", - "item_type": "string", - "Item_desc": "Subject of email sent to miner when worker stops hashing" - }, - { - "id": 72, - "module": "general", - "item": "emailSig", - "item_value": "NodeJS-Pool Administration Team", - "item_type": "string", - "Item_desc": "Signature line for the emails." - }, - { - "id": 73, - "module": "payout", - "item": "timer", - "item_value": "120", - "item_type": "int", - "Item_desc": "Number of minutes between main payment daemon cycles" - }, - { - "id": 74, - "module": "payout", - "item": "timerRetry", - "item_value": "25", - "item_type": "int", - "Item_desc": "Number of minutes between payment daemon retrying due to not enough funds" - }, - { - "id": 75, - "module": "payout", - "item": "priority", - "item_value": "0", - "item_type": "int", - "Item_desc": "Payout priority setting. 0 = use default (4x fee); 1 = low prio (1x fee)" - }, - { - "id": 76, - "module": "payout", - "item": "fee", - "item_value": "10000000000", - "item_type": "int", - "Item_desc": "Atomic units of coin to use ass a fee" - }, - { - "id": 77, - "module": "payout", - "item": "unlock_time", - "item_value": "0", - "item_type": "int", - "Item_desc": "Number of blocks assumed before the payout unlocks." - } -] diff --git a/sql_sync/sql_sync.js b/sql_sync/sql_sync.js deleted file mode 100644 index 72644bbd..00000000 --- a/sql_sync/sql_sync.js +++ /dev/null @@ -1,31 +0,0 @@ -"use strict"; -let mysql = require("promise-mysql"); -let fs = require("fs"); -let config = fs.readFileSync("../config.json"); -let sql_schema = fs.readFileSync("config_entries.json"); -let async = require("async"); - -global.config = JSON.parse(config); -global.mysql = mysql.createPool(global.config.mysql); -global.schema = JSON.parse(sql_schema); - -// Config Table Layout -// . - -let loopCount = 0; -let updatedCount = 0; -async.eachSeries(global.schema, function(entry, callback){ - global.mysql.query("SELECT * FROM config WHERE module = ? AND item = ?", [entry.module, entry.item]).then(function(rows){ - loopCount += 1; - if (rows.length > 0){ - return callback(); - } - updatedCount += 1; - global.mysql.query("INSERT INTO config (module, item, item_value, item_type, Item_desc) VALUES (?, ?, ?, ?, ?)", [entry.module, entry.item, entry.item_value, entry.item_type, entry.Item_desc]).then(function(){ - return callback(); - }); - }); -}, function(){ - console.log("Updated SQL schema with "+updatedCount+" new rows! Exiting!"); - process.exit(); -}); \ No newline at end of file diff --git a/tools/blocks.js b/tools/blocks.js deleted file mode 100644 index 1de41493..00000000 --- a/tools/blocks.js +++ /dev/null @@ -1,44 +0,0 @@ -const valid_actions = ['finder', 'stats', 'list']; -let error = 0; - -if (!global.argv.hasOwnProperty('action') || valid_actions.indexOf(global.argv.action) === -1) { - console.error("No action provided to block module."); - console.error("Valid actions: " + valid_actions.join(', ')); -} - -switch (global.argv.action) { - case 'finder': - if (!global.argv.hasOwnProperty('value')) { - console.error('No block provided in value field. Please use --value=blockID'); - error = 1; - break; - } - let blockID = parseInt(global.argv.value); - let block_data = global.database.getBlockByID(blockID); - /* - required string hash = 1; - required int64 difficulty = 2; - required int64 shares = 3; - required int64 timestamp = 4; - required POOLTYPE poolType = 5; - required bool unlocked = 6; - required bool valid = 7; - optional int64 value = 8; - */ - if (!block_data) { - console.error("Invalid blockID provided."); - error = 1; - break; - } - console.log("Data for block: " + blockID + '\n' + - 'Hash: ' + block_data.hash + '\n' + - 'Difficulty: ' + block_data.hash + '\n' + - 'Hashes Required: ' + block_data.hash + '\n' + - 'Find Time: ' + block_data.hash + '\n' + - 'Pool Type: ' + block_data.hash + '\n' + - 'Unlocked: ' + block_data.hash + '\n' + - 'Valid: ' + block_data.hash + '\n' + - 'Value: ' + block_data.hash + '\n'); -} - -process.exit(error); \ No newline at end of file diff --git a/user_scripts/balance_move.js b/user_scripts/balance_move.js new file mode 100644 index 00000000..9f4033a0 --- /dev/null +++ b/user_scripts/balance_move.js @@ -0,0 +1,96 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.old_user) { + console.error("Please specify old_user address to move balance from"); + process.exit(1); +} +const old_user = argv.old_user; + +if (!argv.new_user) { + console.error("Please specify new_user address to move balance to"); + process.exit(1); +} +const new_user = argv.new_user; + +require("../init_mini.js").init(function() { + const old_parts = old_user.split("."); + const old_address = old_parts.length === 1 ? old_user : old_parts[0]; + const old_payment_id = old_parts.length === 2 ? old_parts[1] : null; + + const new_parts = new_user.split("."); + const new_address = new_parts.length === 1 ? new_user : new_parts[0]; + const new_payment_id = new_parts.length === 2 ? new_parts[1] : null; + + console.log("Old Address: " + old_address); + console.log("Old PaymentID: " + old_payment_id); + console.log("New Address: " + new_address); + console.log("New PaymentID: " + new_payment_id); + + const old_where_str = old_payment_id === null ? "payment_address = '" + old_address + "' AND payment_id IS NULL" + : "payment_address = '" + old_address + "' AND payment_id = '" + old_payment_id + "'"; + + const new_where_str = new_payment_id === null ? "payment_address = '" + new_address + "' AND payment_id IS NULL" + : "payment_address = '" + new_address + "' AND payment_id = '" + new_payment_id + "'"; + + let old_amount; + + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + old_where_str).then(function (rows) { + if (rows.length != 1) { + console.error("Can't find old_user!"); + process.exit(1); + } + old_amount = rows[0].amount; + console.log("Old address amount: " + global.support.coinToDecimal(old_amount)); + console.log("Old address last update time: " + rows[0].last_edited); + if (Date.now()/1000 - global.support.formatDateFromSQL(rows[0].last_edited) < 24*60*60) { + console.error("There was recent amount update. Refusing to continue!"); + process.exit(1); + } + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + new_where_str).then(function (rows) { + if (rows.length != 1) { + console.error("Can't find new_user!"); + process.exit(1); + } + console.log("New address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE balance SET amount = '0' WHERE " + old_where_str).then(function (rows) { + console.log("UPDATE balance SET amount = '0' WHERE " + old_where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE balance SET amount = amount + " + old_amount + " WHERE " + new_where_str).then(function (rows) { + console.log("UPDATE balance SET amount = amount + " + old_amount + " WHERE " + new_where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + old_where_str).then(function (rows) { + console.log("New old address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + new_where_str).then(function (rows) { + console.log("New new address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + console.log("DONE"); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/balance_move_force.js b/user_scripts/balance_move_force.js new file mode 100644 index 00000000..4aefc243 --- /dev/null +++ b/user_scripts/balance_move_force.js @@ -0,0 +1,92 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.old_user) { + console.error("Please specify old_user address to move balance from"); + process.exit(1); +} +const old_user = argv.old_user; + +if (!argv.new_user) { + console.error("Please specify new_user address to move balance to"); + process.exit(1); +} +const new_user = argv.new_user; + +require("../init_mini.js").init(function() { + const old_parts = old_user.split("."); + const old_address = old_parts.length === 1 ? old_user : old_parts[0]; + const old_payment_id = old_parts.length === 2 ? old_parts[1] : null; + + const new_parts = new_user.split("."); + const new_address = new_parts.length === 1 ? new_user : new_parts[0]; + const new_payment_id = new_parts.length === 2 ? new_parts[1] : null; + + console.log("Old Address: " + old_address); + console.log("Old PaymentID: " + old_payment_id); + console.log("New Address: " + new_address); + console.log("New PaymentID: " + new_payment_id); + + const old_where_str = old_payment_id === null ? "payment_address = '" + old_address + "' AND payment_id IS NULL" + : "payment_address = '" + old_address + "' AND payment_id = '" + old_payment_id + "'"; + + const new_where_str = new_payment_id === null ? "payment_address = '" + new_address + "' AND payment_id IS NULL" + : "payment_address = '" + new_address + "' AND payment_id = '" + new_payment_id + "'"; + + let old_amount; + + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + old_where_str).then(function (rows) { + if (rows.length != 1) { + console.error("Can't find old_user!"); + process.exit(1); + } + old_amount = rows[0].amount; + console.log("Old address amount: " + global.support.coinToDecimal(old_amount)); + console.log("Old address last update time: " + rows[0].last_edited); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + new_where_str).then(function (rows) { + if (rows.length != 1) { + console.error("Can't find new_user!"); + process.exit(1); + } + console.log("New address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE balance SET amount = '0' WHERE " + old_where_str).then(function (rows) { + console.log("UPDATE balance SET amount = '0' WHERE " + old_where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE balance SET amount = amount + " + old_amount + " WHERE " + new_where_str).then(function (rows) { + console.log("UPDATE balance SET amount = amount + " + old_amount + " WHERE " + new_where_str); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + old_where_str).then(function (rows) { + console.log("New old address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM balance WHERE " + new_where_str).then(function (rows) { + console.log("New new address amount: " + global.support.coinToDecimal(rows[0].amount)); + callback(); + }); + }, + function (callback) { + console.log("DONE"); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/ban_user.js b/user_scripts/ban_user.js new file mode 100644 index 00000000..4b794353 --- /dev/null +++ b/user_scripts/ban_user.js @@ -0,0 +1,39 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to ban"); + process.exit(1); +} +const user = argv.user; + +if (!argv.reason) { + console.error("Please specify reason to ban"); + process.exit(1); +} +const reason = argv.reason; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query('INSERT INTO bans (mining_address, reason) VALUES (?, ?)', [user, reason]).then(function (rows) { + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM bans").then(function (rows) { + for (let i in rows) { + const row = rows[i]; + console.log(row.mining_address + ": " + row.reason); + } + callback(); + }); + }, + function (callback) { + console.log("Done. User was banned."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/email_disable.js b/user_scripts/email_disable.js new file mode 100644 index 00000000..39d7ed27 --- /dev/null +++ b/user_scripts/email_disable.js @@ -0,0 +1,35 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to set"); + process.exit(1); +} + +const user = argv.user; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length != 1) { + console.error("User password and thus email is not yet set"); + process.exit(1); + } + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE users SET enable_email = '0' WHERE username = ?", [user]).then(function (rows) { + console.log("UPDATE users SET enable_email = '0' WHERE username = " + user); + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/lock_pay.js b/user_scripts/lock_pay.js new file mode 100644 index 00000000..819e7bf8 --- /dev/null +++ b/user_scripts/lock_pay.js @@ -0,0 +1,35 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to set"); + process.exit(1); +} + +const user = argv.user; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length != 1) { + console.error("User password and thus email is not yet set"); + process.exit(1); + } + callback(); + }); + }, + function (callback) { + global.mysql.query("UPDATE users SET payout_threshold_lock = '1' WHERE username = ?", [user]).then(function (rows) { + console.log("UPDATE users SET payout_threshold_lock = '1' WHERE username = " + user); + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/pass_reset.js b/user_scripts/pass_reset.js new file mode 100644 index 00000000..9c92ab71 --- /dev/null +++ b/user_scripts/pass_reset.js @@ -0,0 +1,35 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to delete"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length != 1) { + console.error("Your password is not yet set. To do that you need to set password field in your miner to \":\", where is any name (without : character) and is your password (depending on miner password can be in in command line, config.json or config.txt files). Optionally you can use your email as your password if you want notifications about miner downtimes from the pool. You need to make sure you restart your miner and your miner submits at least one valid share for password to be set."); + process.exit(1); + } + console.log("Found rows in users table: " + rows.length); + callback(); + }); + }, + function (callback) { + global.mysql.query("DELETE FROM users WHERE username = ?", [user]).then(function (rows) { + console.log("DELETE FROM users WHERE username = " + user); + callback(); + }); + }, + function (callback) { + console.log("Done. Please do not forget to restart your miner to apply new password and set payment threshold since it was reset as well"); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/pass_set.js b/user_scripts/pass_set.js new file mode 100644 index 00000000..dbfb7403 --- /dev/null +++ b/user_scripts/pass_set.js @@ -0,0 +1,40 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to set"); + process.exit(1); +} +if (!argv.pass) { + console.error("Please specify user pass to set"); + process.exit(1); +} +const user = argv.user; +const pass = argv.pass; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM users WHERE username = ?", [user]).then(function (rows) { + if (rows.length == 1) { + console.error("Your password is already set, so can not set it again"); + console.log("Found rows in users table: " + rows.length); + process.exit(1); + } + callback(); + }); + }, + function (callback) { + global.mysql.query("INSERT INTO users (username, email, enable_email) VALUES (?, ?, 0)", [user, pass]).then(function (rows) { + console.log("INSERT INTO users (username, email, enable_email) VALUES (" + user + ", " + pass + ", 0)"); + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/pay_set.js b/user_scripts/pay_set.js new file mode 100644 index 00000000..1c559cc8 --- /dev/null +++ b/user_scripts/pay_set.js @@ -0,0 +1,26 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to set"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + const pay = global.support.decimalToCoin(argv.pay ? argv.pay : 0.003); + async.waterfall([ + function (callback) { + global.mysql.query("UPDATE users SET payout_threshold=? WHERE username=?", [pay, user]).then(function (rows) { + console.log("UPDATE users SET payout_threshold=" + pay + " WHERE username=" + user); + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/show_bans.js b/user_scripts/show_bans.js new file mode 100644 index 00000000..f6ad7126 --- /dev/null +++ b/user_scripts/show_bans.js @@ -0,0 +1,21 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query("SELECT * FROM bans").then(function (rows) { + for (let i in rows) { + const row = rows[i]; + console.log(row.mining_address + ": " + row.reason); + } + callback(); + }); + }, + function (callback) { + console.log("Done."); + process.exit(0); + } + ]); +}); diff --git a/user_scripts/unban_user.js b/user_scripts/unban_user.js new file mode 100644 index 00000000..59c4fa73 --- /dev/null +++ b/user_scripts/unban_user.js @@ -0,0 +1,33 @@ +"use strict"; +const mysql = require("promise-mysql"); +const async = require("async"); +const argv = require('minimist')(process.argv.slice(2)); + +if (!argv.user) { + console.error("Please specify user address to unban"); + process.exit(1); +} +const user = argv.user; + +require("../init_mini.js").init(function() { + async.waterfall([ + function (callback) { + global.mysql.query('DELETE FROM bans WHERE mining_address = ?', [user]).then(function (rows) { + callback(); + }); + }, + function (callback) { + global.mysql.query("SELECT * FROM bans").then(function (rows) { + for (let i in rows) { + const row = rows[i]; + console.log(row.mining_address + ": " + row.reason); + } + callback(); + }); + }, + function (callback) { + console.log("Done. User was unbanned."); + process.exit(0); + } + ]); +});