[JENKINS-64383] combined refrepo became our bottleneck, support a fanout location too #644

…RL} to replace by url => funny dir subtree in filesystem

…atibleGitAPIImpl.java so its logic (expected to grow in complexity) can be shared by both JGitAPIImpl.java and CliGitAPIImpl.java

…d ref-repos in submodule checkouts (only CliGitAPIImpl.java has it)

…l String

… findParameterizedReferenceRepository()

…SHA256} reference repos

… name for the mirrored repo

…6() with real sanity checks

…(): do not bother normalizing the URL if the string is not with supported suffix

… test runner

…ame string

…intsToLocal*Mirror() with custom paths and bare vs workspace repos

…6() to find expected paths

…refactor getObjectPath(referencePath) to check on git dirs elsewhere later

…erenceRepository() and isParameterizedReferenceRepository() taking a File reference (not only a String) object

…ceRepository() processing to stderr

…o stderr

… keep original reference intact, and as indicator to recreate referencePath object once for many cases

…256_FALLBACK suffixes for using unsuffixed directory if expanded path points nowhere useful

…ith fallback modes of operation

…d GIT_SUBMODULES_FALLBACK support

…path names

…d GIT_SUBMODULES_FALLBACK support

…SII and for "safe" URLs without username/password, to match jGit version

…note it in assertion stacktrace

…GitUrl() directly

…lf (might have or not the SHA dir after it)

…needed for GIT_SUBMODULES

…odule searches

…earch for "needle" in bare repo case

…in scope

…that we might not-abort after all

…odulesUrls() in findParameterizedReferenceRepository()

…zedReferenceRepository(): deal with LinkedHashSet instead of plain Set to maintain insertion order as the priority

…(): comment a TODO in case there are several hits in the getSubmodulesUrls() return

…f isbare

…ading getObjectsFile(), and varnames inside it too

…ory() fallbacks, quick-check for "<chosenname>.git" dirname existence and usability too

…ory() move GIT_URL_FALLBACK closer to GIT_URL

…ory() add a GIT_URL_BASENAME/GIT_URL_BASENAME_FALLBACK option

…ory() revise the GIT_SUBMODULES/GIT_SUBMODULES_FALLBACK handling

…ameterizedReferenceRepository() to getSubmodulesUrls(), in case users manually cloned refrepos with whatever casing they use throughout

…dParameterizedReferenceRepository() GIT_URL_BASENAME, in case users manually cloned refrepos with whatever casing they use throughout

…s not there yet (might be from *_FALLBACK cases)

…ead of stderr (except cases currently anticipated by unit-tests)

…irnames

…epository() with referenceBaseDir

… actually look at in getSubmodulesUrls()

…y know referenceBaseDir

…f" dirname as the git workspace, not needle

…to inspect other repos with same class as currently used

…eferenceGit

…bs string in case of a hit

…ative dirname string in case of no hit

…lesUrls()

… pathnames (cached hits) more reliably in getSubmodulesUrls()

…bs string in case of a hit

… second phase of needle-searching, and only look for submodule definitions in dirs that ARE git workspaces (avoid looking into parent refrepoBaseDir all the time for simple subdirs)

…for being git repos, twice

…it repos, only (submodules do not host their index and have no "objects" directly inside)

…ting a .git object too

…odule definitions in current dir (might recurse that eventually)

…workspace with an exception

… phase

…cond phase

… outcome of resolving the parameterized Git reference repository, to explain the clutter from "git" calls

…, and move declarations of values derived from nun-null needle

…y logic needed for both search for a needle or full listing

… uriNames pair value a "remoteName"

…rls() if it has "prioritized" pathnames, to recurse into them first

…rch from .gitmodules data

…bmodulesUrls() signature

…n getSubmodulesUrls() right away

…esUrls()

…objects" in getSubmodulesUrls()

…here were no hits, suggest the basename(URL) subdir (friendlier for co-hosting forks)

…with "gitdir" entry, for getObjectsFile()

…common getObjectsFile() instead of direct FS object queries

…rn both a Boolean whether it had an exact match, and the result Set

…does not return "object" dir File named with getCanonicalPath()

…eader why they see lots of "git rev-parse --is-bare-repository" + "git config --local --list" messages

…clarify when "objects" may be not under "reference"

…Urls() should be canonical (thus unique), if possible

…milar* to needle in getSubmodulesUrls() if no exact match happened

…(exists and is a dir) before trying a useless and failure-prone search in findParameterizedReferenceRepository()

…ng GIT_URL_BASENAME

…(): move generic comments about the design from GIT_URL handling to head of function

…(): drop original-experimental support for verbatim GIT_URL as the sub-pathname, due to its little portability

… can return null

…esUrls(): catch generic Throwable, not just Exception, while digging into submodules

…ro handling for caught IOExpection

…" should never return null

…reader with specified encoding

…s (tried as relative) on Windows

…for absolute "/" path in some lines

…xception

…ckout due to not-initialized referenceBaseDir (no git repo there)

…at may take a few git queries to resolve

…ely less on strict wording of messages from git CLI tool

…f messages from git CLI tool (after PR jenkinsci#824 mass renaming)

…MarkEWaite

…MarkEWaite) and fix some Keeping an eye out for comprehensibility of the resulting source code text

…master branch

Promulgates inconsistent coding style and breaks "logical" markup of messages defined on multiple lines (like "item " + javavar) which it fails to keep together, etc. Unavoidable evil I guess, but hopefully someone can configure it.

Use English locale as a UTF-8 capable locale

Signed-off-by: Jim Klimov <jimklimov+jenkinsci@gmail.com>

Commits on Sep 5, 2022

Merge branch 'master' into refrepo-args

MarkEWaite committed Sep 5, 2022

Configuration menu

View commit details

Copy full SHA for 4222f4e

Browse repository at this point

Copy the full SHA

4222f4e View commit details

Browse the repository at this point in the history

Commits on Sep 14, 2022

Merge branch 'master' into refrepo-args

MarkEWaite committed Sep 14, 2022

Configuration menu

View commit details

Copy full SHA for 63ca351

Browse repository at this point

Copy the full SHA

63ca351 View commit details

Browse the repository at this point in the history

Commits on Dec 21, 2022

Merge branch 'master' into refrepo-args

jimklimov committed Dec 21, 2022

Configuration menu

View commit details

Copy full SHA for 2544b99

Browse repository at this point

Copy the full SHA

2544b99 View commit details

Browse the repository at this point in the history

Commits on Jan 17, 2024

Merge branch 'master' into refrepo-args

MarkEWaite committed Jan 17, 2024

Configuration menu

View commit details

Copy full SHA for 9da15fa

Browse repository at this point

Copy the full SHA

9da15fa View commit details

Browse the repository at this point in the history

Commits on Mar 9, 2024

Merge branch 'master' into refrepo-args

MarkEWaite committed Mar 9, 2024

Configuration menu

View commit details

Copy full SHA for 1f379a7

Browse repository at this point

Copy the full SHA

1f379a7 View commit details

Browse the repository at this point in the history

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[JENKINS-64383] combined refrepo became our bottleneck, support a fanout location too #644

[JENKINS-64383] combined refrepo became our bottleneck, support a fanout location too #644

Commits on Jan 10, 2021

Commits on Jan 11, 2021

Commits on Jan 12, 2021

Commits on Jan 13, 2021

Commits on Jan 14, 2021

Commits on Jan 16, 2021

Commits on Jan 17, 2021

Commits on Jan 18, 2021

Commits on Jan 31, 2021

Commits on Feb 22, 2021

Commits on Jan 27, 2022

Commits on Aug 1, 2022

Commits on Aug 10, 2022

Commits on Sep 5, 2022

Commits on Sep 14, 2022

Commits on Dec 21, 2022

Commits on Apr 11, 2023

Commits on Jun 14, 2023

Commits on Sep 13, 2023

Commits on Jan 17, 2024

Commits on Mar 9, 2024

Commits on Jun 14, 2024

[JENKINS-64383] combined refrepo became our bottleneck, support a fanout location too #644

Are you sure you want to change the base?

[JENKINS-64383] combined refrepo became our bottleneck, support a fanout location too #644

Commits on Jan 10, 2021

Commits on Jan 11, 2021

Commits on Jan 12, 2021

Commits on Jan 13, 2021

Commits on Jan 14, 2021

Commits on Jan 16, 2021

Commits on Jan 17, 2021

Commits on Jan 18, 2021

Commits on Jan 31, 2021

Commits on Feb 22, 2021

Commits on Jan 27, 2022

Commits on Aug 1, 2022

Commits on Aug 10, 2022

Commits on Sep 5, 2022

Commits on Sep 14, 2022

Commits on Dec 21, 2022

Commits on Apr 11, 2023

Commits on Jun 14, 2023

Commits on Sep 13, 2023

Commits on Jan 17, 2024

Commits on Mar 9, 2024

Commits on Jun 14, 2024