From 433256b7b29d5ff48128a6a2bacffc379c72cc85 Mon Sep 17 00:00:00 2001
From: mrfry <mr.fry@tutanota.com>
Date: Sun, 9 Apr 2023 07:52:01 +0200
Subject: [PATCH] readme update
---
README.md | 80 +++++++++++++++++++++++++++++++++++++++----------------
1 file changed, 57 insertions(+), 23 deletions(-)
diff --git a/README.md b/README.md
index 3544094..06eb18c 100755
--- a/README.md
+++ b/README.md
@@ -36,7 +36,7 @@ threads, and serve more requests at once. The used cores can be limited with env
## Setup
-Run `./scripts/setup.sh`, then `npm run dev` for development, or `npm run start` for prod
+Run `./scripts/setup.sh`, then `npm run dev` for development, or `npm run start` for prod.
On the first run there will be a number of errors, that some files weren't found. Please create them
according to the messages, these are necessary for the server to function.
@@ -44,6 +44,8 @@ according to the messages, these are necessary for the server to function.
There will be also a lot of information about files and other necessary things being created. **Please
read them very carefully, you should know about what was created!**
+The setup script can be also used to update and rebuild all git submodules.
+
## Web server paths
There are different routes assigned to different modules. You can see these in detail in the files
@@ -52,16 +54,17 @@ There are different routes assigned to different modules. You can see these in d
## Peer to peer
This server implements P2P functionality. It can fetch question databases and users from other
-server instances, and merge the response data to its own databases
+server instances, and merge the response data to its own databases.
To setup P2P functionality you have to create a few files in `./data/p2p`:
* `selfInfo.json`: information of this peer. Required:
```
{
- "name": "Any name you choose",
+ "name": "any name you choose",
"contact": "contact to server administrator (irc server, e-mail, anything)",
- "host": "server host (like somesite.com, without 'http(s)://')",
+ "host": "server host (like somesite.com, without 'http(s):// and /-s')",
+ "pw": "password to the host, so the server can log in there",
"port": "server port, number"
}
```
@@ -69,14 +72,15 @@ To setup P2P functionality you have to create a few files in `./data/p2p`:
* `peers.json` : an array, with objects same as above, and `{ publicKey: "public key of the server"
}`. Public key is used to encrypt the users database in the response, so they can be synced too.
-Extra configuration: HTTP and pw! TODO
-
Uppon syncing data or having a peer request data from your server there will be new entries in
`./data/p2p/thirdPartyPeers.json`. Here you can review the peers, see their contact and host, and if
you choose you can add them to your `peers.json` file. `thirdPartyPeers.json` should also contain
-the public key
+the public key.
-To start syncing user \#1 should perform a get request to `/syncp2pdata`
+To start syncing user \#1 should perform a get request to `/syncp2pdata`.
+
+If the peer doesn't have a https server (only http) then `http: true` should be specified in
+`peers.json`
## Maintenance
@@ -87,7 +91,7 @@ The server doesn't require that much maintenance, but you are advised to:
* Check and moderate the forum(s) of the page
* Sync the server with other peers (this can be automated)
* Check `./data/p2p/thirdPartyPeers.json` file for new peers, and decide if you should use them
- * Regularly check if there is an update to this server and submodules, and update them (using git)
+ * Regularly check if there is an update to this server and git submodules, and update them
* Watch out for directories that can get big:
* `./stats`: server statistics and logs
* `./data/dbs/backup`: backup of databases
@@ -104,6 +108,19 @@ The server doesn't require that much maintenance, but you are advised to:
* `./publicDirs/qminingPublic/savedQuestions`: unanswered questions saved by the userscript
* Most files not tracked by git
+## Server maintenance utils
+
+These scripts can be found in `./src/standaloneUtils`.
+
+ | Name | Description |
+ | --- | --- |
+ | serverMaintenenceUtils.js | Designed to be a single script to collect all these utils. Not done yet |
+ | dbSetup.js | Sets up the database |
+ | rmDuplicates.js | Removes duplicates (questions and subjects) from question db-s. Can merge db-s. This is extremely CPU intensive |
+
+The other undocumented scripts are rather old, and should be checked if they work, and if they are
+needed at all
+
## Server usage statistics
The real time log can be found in `./stats/(v)logs`. Each folder contains rotated logs, one file per
@@ -119,19 +136,6 @@ statistics from the current day. For ex.: if its -4, it displays stats from 4 da
`./scripts/exportStats.js` exports data configured in `exportStats.js`-s first few line to .csv. The
result can be found in the directory it was ran.
-## Server maintenance utils
-
-These scripts can be found in `./src/standaloneUtils`.
-
- | Name | Description |
- | --- | --- |
- | serverMaintenenceUtils.js | Designed to be a single script to collect all these utils. Not done yet |
- | dbSetup.js | Sets up the database |
- | rmDuplicates.js | Removes duplicates (questions and subjects) from question db-s. Can merge db-s. This is extremely CPU intensive |
-
-The other undocumented scripts are rather old, and should be checked if they work, and if they are
-needed at all
-
## Environment variables
| Name | Type | Description |
@@ -155,6 +159,36 @@ needed at all
| npm run test | Runs jest tests |
| npm run test-debug | Runs jest tests in watch mode with debugger attached |
+## Git submodules
+
+Details about the specific git submodule should be found in its directory.
+
+# qmining-page
+
+Next.js frontend for the server. The server serves its static exported html files. The frontend
+contains many features, including chat using sockets, forum, question database viewer and other
+pages detailing information about the userscript
+
+https://gitlab.com/MrFry/qmining-page
+
+# qmining-data-editor
+
+Next.js frontend designed to give the users the ability, to edit the question dbs. Users can add new
+questions, edit existing ones, delete invalid questions and many more. The server saves questions
+and its possible answers during the users are solving online tests. These are saved, and can be
+viewed in the data editor, and the correct answers can be choosen, and saved in the question
+databases.
+
+https://gitlab.com/MrFry/qmining-data-editor
+
+# moodle-test-userscript
+
+Javasript userscript to be run in tampermonkey or similar userscript handler. It's job is to collect
+questions during tests, and on test review pages, and send it to the server. During the script also
+asks the server, if there is a solution saved for the current question, and if yes, it displays it.
+
+https://gitlab.com/MrFry/moodle-test-userscript
+
## Detailed file structure
```
.
@@ -163,7 +197,6 @@ needed at all
│ ├── apiRootRedirectTo url where domain/api should redirect to
│ ├── dbs/ directory for databases, and for their backups
│ ├── domain the domain the server is hosted on
-│ ├── donateURL url where the donate button should take to TODO: check if this is needed
│ ├── f/ user files received TODO: check if this is needed
│ ├── links.json urls for irc, patreon and donate
│ ├── nolog ids of users separated by new lines to ignore on logging
@@ -206,6 +239,7 @@ needed at all
│ ├── qmining-data-editor data editor frontend
│ └── qmining-page qmining frontend
├── testingTools testing tools for the server
+├── defaultPublicFiles static public files that the frontends use, like images
└── publicDirs/ public directories of the server, mostly available on the domain root
└── qminingPublic/ qmining module public path (modules: qmining, dataeditor, api)
├── backs/ question database backups