albert/ncore-leaderboard
1
0
Fork 0
mirror of https://github.com/skidoodle/ncore-leaderboard.git synced 2025-02-15 05:29:15 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions
📁 Scrape and sort profiles from nCore
10 commits 1 branch 0 tags 54 KiB
Find a file
Exact
skidoodle b33bf769f1
Update go.yml
2024-12-31 02:24:41 +01:00
.github/workflows Update go.yml 2024-12-31 02:24:41 +01:00
.env.example first commit 2024-12-31 01:50:12 +01:00
.gitignore first commit 2024-12-31 01:50:12 +01:00
go.mod rename 2024-12-31 02:16:34 +01:00
go.sum update dependencies and improve profile processing with sorting and error handling 2024-12-31 02:01:50 +01:00
license Create license 2024-12-31 02:12:52 +01:00
main.go update dependencies and improve profile processing with sorting and error handling 2024-12-31 02:01:50 +01:00
readme.md Update readme.md 2024-12-31 02:21:46 +01:00

readme.md

nCore Profile Scraper

This is a Go program for scraping and sorting user profile data from nCore, saving results to a CSV file.

Key Features

  • Concurrent Scraping: Fast, parallel processing of profiles.
  • Quicksort Algorithm: Efficient sorting by attributes.
  • Batch Writing: Saves data incrementally to reduce memory usage.

Setup

  1. Clone the repository and install dependencies:

    git clone https://github.com/skidoodle/ncore-leaderboard
cd ncore-leaderboard
go mod tidy

  2. Create a .env file with your credentials:

    NICK=your_username
PASS=your_pass

Usage

Run the scraper: bash go run main.go

  • Scrapes profiles from the configured range.
  • Outputs sorted data to output.log in CSV format.

Configuration

Edit these parameters in main.go as needed:

startProfile, endProfile: Profile ID range.
concurrency: Number of concurrent requests.
outputFile: Output file name.
writeBatch: Profiles processed per save.

Output Format

The CSV file output.log contains:

  1. Profile URL
  2. Attribute Value (e.g., rank)

License

This project is licensed under the GPL-3.0 License.