1 // Copyright 2018 The Go Authors. All rights reserved.
2 // Use of this source code is governed by a BSD-style
3 // license that can be found in the LICENSE file.
4 5 /*
6 Package packages loads Go packages for inspection and analysis.
7 8 The [Load] function takes as input a list of patterns and returns a
9 list of [Package] values describing individual packages matched by those
10 patterns.
11 A [Config] specifies configuration options, the most important of which is
12 the [LoadMode], which controls the amount of detail in the loaded packages.
13 14 Load passes most patterns directly to the underlying build tool.
15 The default build tool is the go command.
16 Its supported patterns are described at
17 https://pkg.go.dev/cmd/go#hdr-Package_lists_and_patterns.
18 Other build systems may be supported by providing a "driver";
19 see [The driver protocol].
20 21 All patterns with the prefix "query=", where query is a
22 non-empty string of letters from [a-z], are reserved and may be
23 interpreted as query operators.
24 25 Two query operators are currently supported: "file" and "pattern".
26 27 The query "file=path/to/file.go" matches the package or packages enclosing
28 the Go source file path/to/file.go. For example "file=~/go/src/fmt/print.go"
29 might return the packages "fmt" and "fmt [fmt.test]".
30 31 The query "pattern=string" causes "string" to be passed directly to
32 the underlying build tool. In most cases this is unnecessary,
33 but an application can use Load("pattern=" + x) as an escaping mechanism
34 to ensure that x is not interpreted as a query operator if it contains '='.
35 36 All other query operators are reserved for future use and currently
37 cause Load to report an error.
38 39 The Package struct provides basic information about the package, including
40 41 - ID, a unique identifier for the package in the returned set;
42 - GoFiles, the names of the package's Go source files;
43 - Imports, a map from source import strings to the Packages they name;
44 - Types, the type information for the package's exported symbols;
45 - Syntax, the parsed syntax trees for the package's source code; and
46 - TypesInfo, the result of a complete type-check of the package syntax trees.
47 48 (See the documentation for type Package for the complete list of fields
49 and more detailed descriptions.)
50 51 For example,
52 53 Load(nil, "bytes", "unicode...")
54 55 returns four Package structs describing the standard library packages
56 bytes, unicode, unicode/utf16, and unicode/utf8. Note that one pattern
57 can match multiple packages and that a package might be matched by
58 multiple patterns: in general it is not possible to determine which
59 packages correspond to which patterns.
60 61 Note that the list returned by Load contains only the packages matched
62 by the patterns. Their dependencies can be found by walking the import
63 graph using the Imports fields.
64 65 The Load function can be configured by passing a pointer to a Config as
66 the first argument. A nil Config is equivalent to the zero Config, which
67 causes Load to run in [LoadFiles] mode, collecting minimal information.
68 See the documentation for type Config for details.
69 70 As noted earlier, the Config.Mode controls the amount of detail
71 reported about the loaded packages. See the documentation for type LoadMode
72 for details.
73 74 Most tools should pass their command-line arguments (after any flags)
75 uninterpreted to Load, so that it can interpret them
76 according to the conventions of the underlying build system.
77 78 See the Example function for typical usage.
79 See also [golang.org/x/tools/go/packages/internal/linecount]
80 for an example application.
81 82 # The driver protocol
83 84 Load may be used to load Go packages even in Go projects that use
85 alternative build systems, by installing an appropriate "driver"
86 program for the build system and specifying its location in the
87 GOPACKAGESDRIVER environment variable.
88 For example,
89 https://github.com/bazelbuild/rules_go/wiki/Editor-and-tool-integration
90 explains how to use the driver for Bazel.
91 92 The driver program is responsible for interpreting patterns in its
93 preferred notation and reporting information about the packages that
94 those patterns identify. Drivers must also support the special "file="
95 and "pattern=" patterns described above.
96 97 The patterns are provided as positional command-line arguments. A
98 JSON-encoded [DriverRequest] message providing additional information
99 is written to the driver's standard input. The driver must write a
100 JSON-encoded [DriverResponse] message to its standard output. (This
101 message differs from the JSON schema produced by 'go list'.)
102 103 The value of the PWD environment variable seen by the driver process
104 is the preferred name of its working directory. (The working directory
105 may have other aliases due to symbolic links; see the comment on the
106 Dir field of [exec.Cmd] for related information.)
107 When the driver process emits in its response the name of a file
108 that is a descendant of this directory, it must use an absolute path
109 that has the value of PWD as a prefix, to ensure that the returned
110 filenames satisfy the original query.
111 */
112 package packages // import "golang.org/x/tools/go/packages"
113 114 /*
115 116 Motivation and design considerations
117 118 The new package's design solves problems addressed by two existing
119 packages: go/build, which locates and describes packages, and
120 golang.org/x/tools/go/loader, which loads, parses and type-checks them.
121 The go/build.Package structure encodes too much of the 'go build' way
122 of organizing projects, leaving us in need of a data type that describes a
123 package of Go source code independent of the underlying build system.
124 We wanted something that works equally well with go build and vgo, and
125 also other build systems such as Bazel and Blaze, making it possible to
126 construct analysis tools that work in all these environments.
127 Tools such as errcheck and staticcheck were essentially unavailable to
128 the Go community at Google, and some of Google's internal tools for Go
129 are unavailable externally.
130 This new package provides a uniform way to obtain package metadata by
131 querying each of these build systems, optionally supporting their
132 preferred command-line notations for packages, so that tools integrate
133 neatly with users' build environments. The Metadata query function
134 executes an external query tool appropriate to the current workspace.
135 136 Loading packages always returns the complete import graph "all the way down",
137 even if all you want is information about a single package, because the query
138 mechanisms of all the build systems we currently support ({go,vgo} list, and
139 blaze/bazel aspect-based query) cannot provide detailed information
140 about one package without visiting all its dependencies too, so there is
141 no additional asymptotic cost to providing transitive information.
142 (This property might not be true of a hypothetical 5th build system.)
143 144 In calls to TypeCheck, all initial packages, and any package that
145 transitively depends on one of them, must be loaded from source.
146 Consider A->B->C->D->E: if A,C are initial, A,B,C must be loaded from
147 source; D may be loaded from export data, and E may not be loaded at all
148 (though it's possible that D's export data mentions it, so a
149 types.Package may be created for it and exposed.)
150 151 The old loader had a feature to suppress type-checking of function
152 bodies on a per-package basis, primarily intended to reduce the work of
153 obtaining type information for imported packages. Now that imports are
154 satisfied by export data, the optimization no longer seems necessary.
155 156 Despite some early attempts, the old loader did not exploit export data,
157 instead always using the equivalent of WholeProgram mode. This was due
158 to the complexity of mixing source and export data packages (now
159 resolved by the upward traversal mentioned above), and because export data
160 files were nearly always missing or stale. Now that 'go build' supports
161 caching, all the underlying build systems can guarantee to produce
162 export data in a reasonable (amortized) time.
163 164 Test "main" packages synthesized by the build system are now reported as
165 first-class packages, avoiding the need for clients (such as go/ssa) to
166 reinvent this generation logic.
167 168 One way in which go/packages is simpler than the old loader is in its
169 treatment of in-package tests. In-package tests are packages that
170 consist of all the files of the library under test, plus the test files.
171 The old loader constructed in-package tests by a two-phase process of
172 mutation called "augmentation": first it would construct and type check
173 all the ordinary library packages and type-check the packages that
174 depend on them; then it would add more (test) files to the package and
175 type-check again. This two-phase approach had four major problems:
176 1) in processing the tests, the loader modified the library package,
177 leaving no way for a client application to see both the test
178 package and the library package; one would mutate into the other.
179 2) because test files can declare additional methods on types defined in
180 the library portion of the package, the dispatch of method calls in
181 the library portion was affected by the presence of the test files.
182 This should have been a clue that the packages were logically
183 different.
184 3) this model of "augmentation" assumed at most one in-package test
185 per library package, which is true of projects using 'go build',
186 but not other build systems.
187 4) because of the two-phase nature of test processing, all packages that
188 import the library package had to be processed before augmentation,
189 forcing a "one-shot" API and preventing the client from calling Load
190 in several times in sequence as is now possible in WholeProgram mode.
191 (TypeCheck mode has a similar one-shot restriction for a different reason.)
192 193 Early drafts of this package supported "multi-shot" operation.
194 Although it allowed clients to make a sequence of calls (or concurrent
195 calls) to Load, building up the graph of Packages incrementally,
196 it was of marginal value: it complicated the API
197 (since it allowed some options to vary across calls but not others),
198 it complicated the implementation,
199 it cannot be made to work in Types mode, as explained above,
200 and it was less efficient than making one combined call (when this is possible).
201 Among the clients we have inspected, none made multiple calls to load
202 but could not be easily and satisfactorily modified to make only a single call.
203 However, applications changes may be required.
204 For example, the ssadump command loads the user-specified packages
205 and in addition the runtime package. It is tempting to simply append
206 "runtime" to the user-provided list, but that does not work if the user
207 specified an ad-hoc package such as [a.go b.go].
208 Instead, ssadump no longer requests the runtime package,
209 but seeks it among the dependencies of the user-specified packages,
210 and emits an error if it is not found.
211 212 Questions & Tasks
213 214 - Add GOARCH/GOOS?
215 They are not portable concepts, but could be made portable.
216 Our goal has been to allow users to express themselves using the conventions
217 of the underlying build system: if the build system honors GOARCH
218 during a build and during a metadata query, then so should
219 applications built atop that query mechanism.
220 Conversely, if the target architecture of the build is determined by
221 command-line flags, the application can pass the relevant
222 flags through to the build system using a command such as:
223 myapp -query_flag="--cpu=amd64" -query_flag="--os=darwin"
224 However, this approach is low-level, unwieldy, and non-portable.
225 GOOS and GOARCH seem important enough to warrant a dedicated option.
226 227 - How should we handle partial failures such as a mixture of good and
228 malformed patterns, existing and non-existent packages, successful and
229 failed builds, import failures, import cycles, and so on, in a call to
230 Load?
231 232 - Support bazel, blaze, and go1.10 list, not just go1.11 list.
233 234 - Handle (and test) various partial success cases, e.g.
235 a mixture of good packages and:
236 invalid patterns
237 nonexistent packages
238 empty packages
239 packages with malformed package or import declarations
240 unreadable files
241 import cycles
242 other parse errors
243 type errors
244 Make sure we record errors at the correct place in the graph.
245 246 - Missing packages among initial arguments are not reported.
247 Return bogus packages for them, like golist does.
248 249 - "undeclared name" errors (for example) are reported out of source file
250 order. I suspect this is due to the breadth-first resolution now used
251 by go/types. Is that a bug? Discuss with gri.
252 253 */
254