diff --git a/README.md b/README.md index 3544094..06eb18c 100755 --- a/README.md +++ b/README.md @@ -36,7 +36,7 @@ threads, and serve more requests at once. The used cores can be limited with env ## Setup -Run `./scripts/setup.sh`, then `npm run dev` for development, or `npm run start` for prod +Run `./scripts/setup.sh`, then `npm run dev` for development, or `npm run start` for prod. On the first run there will be a number of errors, that some files weren't found. Please create them according to the messages, these are necessary for the server to function. @@ -44,6 +44,8 @@ according to the messages, these are necessary for the server to function. There will be also a lot of information about files and other necessary things being created. **Please read them very carefully, you should know about what was created!** +The setup script can be also used to update and rebuild all git submodules. + ## Web server paths There are different routes assigned to different modules. You can see these in detail in the files @@ -52,16 +54,17 @@ There are different routes assigned to different modules. You can see these in d ## Peer to peer This server implements P2P functionality. It can fetch question databases and users from other -server instances, and merge the response data to its own databases +server instances, and merge the response data to its own databases. To setup P2P functionality you have to create a few files in `./data/p2p`: * `selfInfo.json`: information of this peer. Required: ``` { - "name": "Any name you choose", + "name": "any name you choose", "contact": "contact to server administrator (irc server, e-mail, anything)", - "host": "server host (like somesite.com, without 'http(s)://')", + "host": "server host (like somesite.com, without 'http(s):// and /-s')", + "pw": "password to the host, so the server can log in there", "port": "server port, number" } ``` @@ -69,14 +72,15 @@ To setup P2P functionality you have to create a few files in `./data/p2p`: * `peers.json` : an array, with objects same as above, and `{ publicKey: "public key of the server" }`. Public key is used to encrypt the users database in the response, so they can be synced too. -Extra configuration: HTTP and pw! TODO - Uppon syncing data or having a peer request data from your server there will be new entries in `./data/p2p/thirdPartyPeers.json`. Here you can review the peers, see their contact and host, and if you choose you can add them to your `peers.json` file. `thirdPartyPeers.json` should also contain -the public key +the public key. -To start syncing user \#1 should perform a get request to `/syncp2pdata` +To start syncing user \#1 should perform a get request to `/syncp2pdata`. + +If the peer doesn't have a https server (only http) then `http: true` should be specified in +`peers.json` ## Maintenance @@ -87,7 +91,7 @@ The server doesn't require that much maintenance, but you are advised to: * Check and moderate the forum(s) of the page * Sync the server with other peers (this can be automated) * Check `./data/p2p/thirdPartyPeers.json` file for new peers, and decide if you should use them - * Regularly check if there is an update to this server and submodules, and update them (using git) + * Regularly check if there is an update to this server and git submodules, and update them * Watch out for directories that can get big: * `./stats`: server statistics and logs * `./data/dbs/backup`: backup of databases @@ -104,6 +108,19 @@ The server doesn't require that much maintenance, but you are advised to: * `./publicDirs/qminingPublic/savedQuestions`: unanswered questions saved by the userscript * Most files not tracked by git +## Server maintenance utils + +These scripts can be found in `./src/standaloneUtils`. + + | Name | Description | + | --- | --- | + | serverMaintenenceUtils.js | Designed to be a single script to collect all these utils. Not done yet | + | dbSetup.js | Sets up the database | + | rmDuplicates.js | Removes duplicates (questions and subjects) from question db-s. Can merge db-s. This is extremely CPU intensive | + +The other undocumented scripts are rather old, and should be checked if they work, and if they are +needed at all + ## Server usage statistics The real time log can be found in `./stats/(v)logs`. Each folder contains rotated logs, one file per @@ -119,19 +136,6 @@ statistics from the current day. For ex.: if its -4, it displays stats from 4 da `./scripts/exportStats.js` exports data configured in `exportStats.js`-s first few line to .csv. The result can be found in the directory it was ran. -## Server maintenance utils - -These scripts can be found in `./src/standaloneUtils`. - - | Name | Description | - | --- | --- | - | serverMaintenenceUtils.js | Designed to be a single script to collect all these utils. Not done yet | - | dbSetup.js | Sets up the database | - | rmDuplicates.js | Removes duplicates (questions and subjects) from question db-s. Can merge db-s. This is extremely CPU intensive | - -The other undocumented scripts are rather old, and should be checked if they work, and if they are -needed at all - ## Environment variables | Name | Type | Description | @@ -155,6 +159,36 @@ needed at all | npm run test | Runs jest tests | | npm run test-debug | Runs jest tests in watch mode with debugger attached | +## Git submodules + +Details about the specific git submodule should be found in its directory. + +# qmining-page + +Next.js frontend for the server. The server serves its static exported html files. The frontend +contains many features, including chat using sockets, forum, question database viewer and other +pages detailing information about the userscript + +https://gitlab.com/MrFry/qmining-page + +# qmining-data-editor + +Next.js frontend designed to give the users the ability, to edit the question dbs. Users can add new +questions, edit existing ones, delete invalid questions and many more. The server saves questions +and its possible answers during the users are solving online tests. These are saved, and can be +viewed in the data editor, and the correct answers can be choosen, and saved in the question +databases. + +https://gitlab.com/MrFry/qmining-data-editor + +# moodle-test-userscript + +Javasript userscript to be run in tampermonkey or similar userscript handler. It's job is to collect +questions during tests, and on test review pages, and send it to the server. During the script also +asks the server, if there is a solution saved for the current question, and if yes, it displays it. + +https://gitlab.com/MrFry/moodle-test-userscript + ## Detailed file structure ``` . @@ -163,7 +197,6 @@ needed at all │ ├── apiRootRedirectTo url where domain/api should redirect to │ ├── dbs/ directory for databases, and for their backups │ ├── domain the domain the server is hosted on -│ ├── donateURL url where the donate button should take to TODO: check if this is needed │ ├── f/ user files received TODO: check if this is needed │ ├── links.json urls for irc, patreon and donate │ ├── nolog ids of users separated by new lines to ignore on logging @@ -206,6 +239,7 @@ needed at all │ ├── qmining-data-editor data editor frontend │ └── qmining-page qmining frontend ├── testingTools testing tools for the server +├── defaultPublicFiles static public files that the frontends use, like images └── publicDirs/ public directories of the server, mostly available on the domain root └── qminingPublic/ qmining module public path (modules: qmining, dataeditor, api) ├── backs/ question database backups