[cpm, tools]: Ability to override and set custom hosts (like a self hosted instance/git-daemon) #2837
Closed
Lizzie
wants to merge 3 commits from
selfhost0 into master
pull from: selfhost0
merge into: eden-emu:master
eden-emu:master
eden-emu:xbzk/unreal-unsafe-junk-guards
eden-emu:2ndgpuopts
eden-emu:texture_cache
eden-emu:shader_recompiler
eden-emu:camillelavey-patch-3
eden-emu:fix-grid-autoalign
eden-emu:lizzie/mlp223
eden-emu:work/riscv-eh-impl
eden-emu:bump-httplib-minver
eden-emu:work/httplib-sep-mod
eden-emu:lizzie/ios-port-sud
eden-emu:lizzie/unity-build
eden-emu:lizzie/remove-mlp-2-level
eden-emu:lizzie/lrunsdfkdsjfl
eden-emu:lizzie/mark-bl-dynarmic
eden-emu:lizzie/jit-addresschecks
eden-emu:lizzie/nce-invalidate-split
eden-emu:mmap-fixews
eden-emu:lizzie/readonly-listicons
eden-emu:lizzie/update-faq-link-wwa
eden-emu:xbzk/flicker-fix
eden-emu:xbzk/dma-step-ondemand-flush
eden-emu:ffmpeg-cross-compile
eden-emu:lizzie/stupid-socket-bullshit
eden-emu:lizzie/nuke-vtable-shit
eden-emu:lizzie/ulaunch-attempt1
eden-emu:lizzie/sm-AtmosphereHasService
eden-emu:lizzie/ltofix123
eden-emu:lizzie/macos-vk-metal-fix-stype
eden-emu:lizzie/sgsr
eden-emu:lizzie/inline-123
eden-emu:lizzie/bsdsockets-mod-fix
eden-emu:lizzie/dynarmic-next-2026-03-26
eden-emu:dynarmic-ppc64
eden-emu:lizzie/anv-cached-mesa-hack
eden-emu:lizize/fixcrashwehenimagesizeexceed
eden-emu:lizzie/product-model-wawa
eden-emu:lizzie/multicore-macos-fix1
eden-emu:lizzie/xcode-evil-shit-123
eden-emu:lizzie/draw-state-inline
eden-emu:lizzie/noexcept-dynarmic
eden-emu:memory_changes
eden-emu:lizzie/hleinlineremoveredundant
eden-emu:lizzie/wstring-uni-123
eden-emu:lizzie/stable-shader-pools
eden-emu:feat/22.0.0
eden-emu:lizzie/attempt2-better-list-dynarmic
eden-emu:lizzie/lanczos-opt
eden-emu:lizzie/dynarmic-faster-xbyak
eden-emu:xbzk/vsync-immed-fallback-fix
eden-emu:eden-orbis-ps4
eden-emu:Compatlist
eden-emu:lizzie/macos-network-ifaces
eden-emu:xbzk/input-delay-fix
eden-emu:lizzie/unaligned-attempt-2
eden-emu:vkexperiments1
eden-emu:lizzie/fix-nvmap-handles
eden-emu:lizzie/fix-nvmap-handles-but-evil
eden-emu:dmnt2
eden-emu:eds_changes_phasewise
eden-emu:eds_changes1
eden-emu:lizzie/inline-cmif-request
eden-emu:lizzie/fs-msvc-succks
eden-emu:xbzk/press-back-again-issue-fix
eden-emu:xbzk/background-support
eden-emu:vkfixes1
eden-emu:lizzie/vkexperiments1-highp-fucked
eden-emu:lizzie/edit-handheld-mode
eden-emu:techno48473719
eden-emu:lizzie/invert-gyro
eden-emu:lizzie/restore-fcsm
eden-emu:lines
eden-emu:msvc
eden-emu:kosmickrisp
eden-emu:xbzk/settings-fragment-rework
eden-emu:kosmic_krisp
eden-emu:xbzk-dma-pusher-step-redesign
eden-emu:lizzie/batched-draws-wip
eden-emu:feat/new_lang
eden-emu:vuid02999
eden-emu:vuid04553
eden-emu:lizzie/xbyak-unor-mapfix-with-boost
eden-emu:revert-xbyak
eden-emu:vuid00336_1
eden-emu:lizzie/splay
eden-emu:lizzie/spscs-mpscs-queue
eden-emu:lizzie/settings-blocked-domains
eden-emu:00336
eden-emu:dravee/remove-async
eden-emu:astc_optimizations
eden-emu:bcn-ternary-soft
eden-emu:pipelinederivative
eden-emu:civa
eden-emu:vk-fix-oom-force-maller-buffers
eden-emu:lizzie/macos-fix
eden-emu:camillelavey-patch-1
eden-emu:xbzk/vulkan-vuid-goodies-pack
eden-emu:release-early-fences
eden-emu:gpuopts
eden-emu:lizzie/nce-port1123
eden-emu:lizzie/fix-rw-garten
eden-emu:vk-symph
eden-emu:feat/android_kotlin_compose
eden-emu:descriptor_pool_opt
eden-emu:lizzie/evil-f32-u32-format-fix
eden-emu:liz-dynarmic-macos-fbsd-port
eden-emu:uma
eden-emu:feat/game_override
eden-emu:revert-noinline
eden-emu:liz-get-rid-of-mcl-intrusive-list
eden-emu:querybugfix
eden-emu:qcomopts2
eden-emu:workgroup
eden-emu:lizzie/correcter-convert-abgr8-srgb-d24s8
eden-emu:lizzie/vids-ratatata
eden-emu:lizzie/maxwell-dumb-tools
eden-emu:lizzie/astc-mp4-improv
eden-emu:astc_optimization1
eden-emu:lizzie/gutter-sw-blitter
eden-emu:release/0.1.0
eden-emu:lizzie/qt-fix-user-dialogue
eden-emu:lizzie/fix-my-fuckup-with-vic
eden-emu:vulkanasync
eden-emu:lizzie/fix-dragonslayer-armnce
eden-emu:lizzie/try-fix-win11-crash-fuck
eden-emu:lizzie/adreno5-mali-driver-fix
eden-emu:vulkan-thingy
eden-emu:descriptor
eden-emu:cyber/async-gpu-android
eden-emu:lizzie/inline-dynarmic-spooky
eden-emu:lizzie/audio-remove-recursive-lock
eden-emu:lizzie/rasterizer-recursive-mutex-not
eden-emu:lizzie/texture-pass-revert-1
eden-emu:fix/socket
eden-emu:revertrevert
eden-emu:xbzk/bindless-textures-support
eden-emu:spvopts
eden-emu:sgsrtry
eden-emu:lock-term-1
eden-emu:lizzie/fix-vic-msvc-attempt-2
eden-emu:fs-try
eden-emu:eds-true-adreno-fixes
eden-emu:true-eds
eden-emu:Update-some-barriers-test
eden-emu:macroify-surface-stuffs
eden-emu:test2
eden-emu:eds-true-adreno-fixes-pre-0.1.0
eden-emu:true-eds-pre-0.0.1
eden-emu:woa-turnip-expr
eden-emu:release/0.0.4
eden-emu:interval-zero
eden-emu:sjkdbsdfjkbsdf-2834
eden-emu:mutliplayer-filter-better1
eden-emu:lanobu
eden-emu:true-eds-graphics
eden-emu:xbzk-saf-recursive-write-with-permission-request
eden-emu:refactoreds2
eden-emu:showcase2
eden-emu:showcase
eden-emu:test-revert-gpu-optim
eden-emu:flatopsfixes23485
eden-emu:discfix
eden-emu:fix/discord-rpc
eden-emu:liz-crash-dumps-solaris
eden-emu:sured-revert
eden-emu:atomicops-mxwell
eden-emu:revert-2695
eden-emu:descriptor_set
eden-emu:n64
eden-emu:release/0.0.3
eden-emu:Kernel
eden-emu:nce_cpp
No reviewers
Labels
Clear labels
Specific to android platform
Something is not working
For changes to the ci/building
Used when testers confirm a bug is reported and it is reproducable.
For core changes to the emulator
Specific to desktop PCs, handhelds, etc.
This issue or pull request already exists
New feature
Frontend changes
Changes related to firmware
For enhancements or changes regarding git.
For changes to any of the loader implementations
For use with changes that involve the emulator GPU and any services.
Used on anything Vulkan-related
android
Specific to android platform
audio
bug
Something is not working
ci
For changes to the ci/building
cmake
confirmed
Used when testers confirm a bug is reported and it is reproducable.
core
For core changes to the emulator
desktop
Specific to desktop PCs, handhelds, etc.
duplicate
This issue or pull request already exists
enhancement
New feature
frontend
Frontend changes
fw
Changes related to firmware
git
For enhancements or changes regarding git.
graphics
loader
For changes to any of the loader implementations
nv gpu
For use with changes that involve the emulator GPU and any services.
ui
vulkan
Used on anything Vulkan-related
No labels
android
audio
bug
ci
cmake
confirmed
core
desktop
duplicate
enhancement
frontend
fw
git
graphics
loader
nv gpu
ui
vulkan
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set.
Reference
eden-emu/eden!2837
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "selfhost0"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Signed-off-by: lizzie lizzie@eden-emu.dev
@ -14,2 +14,4 @@"Force system packages for all CPM dependencies (NOT RECOMMENDED)" OFF)option(CPMUTIL_DEFAULT_HOST"Sets the default host when 'git_host' isn't defined" "github.com")https://cmake.org/cmake/help/latest/command/set.html#cache-legacy
set(CPMUTIL_DEFAULT_HOST github.com CACHE STRING "Sets the default host when no custom Git host isn't defined")Though I already have this and a few other changes stashed in CPMUtil, so I will push that whenever it's ready
@ -33,1 +33,4 @@- [Translation Scripts](./translations)## Mirror- [Mirroring scripts](./mirror)Prefer newlines after headings, some Markdown viewers need it
@ -0,0 +12,4 @@echo "1 - Do initial clone/world"echo "2 - Clone submodules (update)"echo "3 - Force manual remotes update (please use crontab instead)"read -p "Selection? [none]: " ANSWERinteractive CLIs are not the greatest. Instead we should use getopt-like parsing
https://git.crueter.xyz/scripts/windev/src/branch/master/install.sh#L379
Also, this is less relevant for a getopt-like solution, but prefer cat-EOF heredocs unless there are a ton of variables/command outputs
@ -0,0 +20,4 @@REPOS=$(find "$CPM_DIR" "$CPM_DIR/src" -maxdepth 3 -type f -name 'cpmfile.json' -not -path 'build' \| sort | uniq | xargs jq -s 'reduce .[] as $item ({}; . * $item)' \| jq -r 'reduce .[] as $i (""; . + (if $i.git_host == null then "https://github.com" else "https://" + $i.git_host end) + "/" + $i.repo + " ")' \| tr ' ' '\n' | xargs -I {} echo {})tools/cpm/common.shwill aggregate everything for you@ -0,0 +31,4 @@cd $TARGET_ROOT_DIR && find . -maxdepth 2 -type d -name '*.git' -print0 \| xargs -0 -I {} sh -c ". $CPM_DIR/tools/mirror/common.sh && git_retrieve_gitmodules {}" \| while IFS= read -r line; doNAME=$(echo "$line" | awk -F'/' '{print $5}')cutis preferred here. Awk is generally reserved for more complex cases, since cut is faster@ -0,0 +38,4 @@donesudo chmod 755 $TARGET_ROOT_DIRelif [ "$ANSWER" = "3" ]; thenecho Forcing git updatesecho always prefers quotes
@ -0,0 +3,4 @@# SPDX-License-Identifier: GPL-3.0-or-later# You must run this at the root of the eden git repoCPM_DIR=$PWDNPROC=`nproc`This will not work on OpenBSD, macOS and a few others
@ -0,0 +29,4 @@case "$1" in--path)shiftSCM_ROOT_DIR=$1This will have unintended behavior if the user specifies path afterwards. Prefer what I did in windev/install.sh; set environment variables to determine which ops to do, then parse them later. This also lets you force ordering and such
fb08764372af65ea9a1eaf65ea9a1e9662b005d8This has the slightly unfortunate side effect of not working exactly as expected, since CPM specifically works with source archives, releases, and tags, not necessarily raw commits from a Git repo--if there is a cgit-like service that has this functionality though, it might work
@crueter wrote in #2837 (comment):
git itself can make a headless git server; cgit is just a frontend - I expect the mirror-er to know that, atleast
cgit offers tarballs (for tags only) btw
<cgit-url>/<path-to-repo>/snapshot/<repo name>-<tag>.tar.gzhttp://example.com/OpenTTD/OpenTTD.git/snapshot/OpenTTD-15.0-beta3.tar.gzThat isn't sufficient, because we need general "commitish" archives; that is, commits, tags, and (theoretically) branches. Not to mention releases. Gentoo's gitweb has something similar, e.g. https://gitweb.gentoo.org/repo/proj/guru.git/snapshot/guru-278b5cb7e54fb4f0a1169042fa3559c5811da581.tar.gz, but that still isn't touching releases.
9662b005d84da9c6303b@crueter check if there are still changes necessary
This isn't in scope of CPMUtil at all. You would have to mirror release artifacts and ALL commitish archives. If you really need to go through the horrifically unnecessary step of mirroring every single dependency (because Boost is really in danger of getting DMCA'ed, right...), just use Forgejo's API and change github.com to git.yourhost.com. I'm not adding this as an option.
Pull request closed