Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make SROA pass more aggressive + make SSA use counting more accurate #44557

Merged
merged 1 commit into from
Mar 21, 2022

Conversation

ianatol
Copy link
Member

@ianatol ianatol commented Mar 11, 2022

Rebase of #44494 + removal of a couple times where we miscount compact.used_ssas

Comment on lines -374 to 375
isa(val, SSAValue) && (compact.used_ssas[val.id] += 1)
return_value = SSAValue(idx′)
inline_compact[idx′] = val
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

inline_compact[idx′] = val calls count_added_node!, which increments inline_compact.used_ssas, which === compact.used_ssas, so incrementing here is incorrect

Comment on lines -431 to -437
for i = 1:length(pn.values)
isassigned(pn.values, i) || continue
v = pn.values[i]
if isa(v, SSAValue)
compact.used_ssas[v.id] += 1
end
end
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, if length(pn.edges) != 1, we call insert_node_here! on NewInstruction(pn..., which does this same incrementing. In the case where we just return pn.values[1], I'm not sure if there are paths that result in this getting counted twice, but we can at least make this change for now

@ianatol
Copy link
Member Author

ianatol commented Mar 11, 2022

@nanosoldier runtests(ALL, vs = ":master")

base/compiler/ssair/ir.jl Outdated Show resolved Hide resolved
@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@ianatol ianatol marked this pull request as ready for review March 14, 2022 21:38
Comment on lines +786 to 788
compact[idx] = nothing
compact[idx] = form_new_preserves(stmt, preserved, new_preserves)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since compact[idx] === stmt, we first need to first set compact[idx] to nothing, because in-place modification can lead to miscounts by setindex.

Comment on lines +449 to +451
compact[leaf] = nothing
stmt.args[argidx] = lifted
compact[leaf] = stmt
Copy link
Member Author

@ianatol ianatol Mar 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we forgot to insert, but we also must use the compact[i] = nothing pattern to avoid miscounting

@ianatol
Copy link
Member Author

ianatol commented Mar 15, 2022

@nanosoldier runtests(ALL, vs = ":master")

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@Keno
Copy link
Member

Keno commented Mar 15, 2022

Looks like most of the failures have the same backtrace, so probably look at those first and then maybe look at Transducers after.

base/compiler/ssair/passes.jl Outdated Show resolved Hide resolved
@ianatol
Copy link
Member Author

ianatol commented Mar 15, 2022

This should pass all CI, and most PkgEval, still working on the issue with Transducers though

@ianatol
Copy link
Member Author

ianatol commented Mar 15, 2022

@nanosoldier runtests(ALL, vs = ":master")

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@Keno
Copy link
Member

Keno commented Mar 16, 2022

Some of those failures are likely to be real. Let's re-run to make sure we get rid of any stochastic failures, but afterwards, we probably need to go through and investigate the failures:

@nanosoldier runtests(["AlgebraPDF", "BSeries", "BayesianQuadrature", "BestApproximation", "ClusterManagers", "DLPack", "Dagger", "DifferenceEquations", "Evolutionary", "FHIRClient", "FLoops", "FlashWeave", "GeoClustering", "GeoDatasets", "GeometricFlux", "GraphPlot", "Limbdark", "ModalIntervalArithmetic", "NeuralQuantumState", "OMETIFF", "OteraEngine", "PackageCompiler", "PhyloPlots", "Pluto", "STREAMBenchmark", "TensorBoardLogger", "Transducers", "VCFTools", "VoronoiGraph"], vs = ":master")

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@Keno
Copy link
Member

Keno commented Mar 16, 2022

BestApproximation has showed up in all the reports, so I'd probably start there.

@ianatol
Copy link
Member Author

ianatol commented Mar 17, 2022

@nanosoldier runtests(ALL, vs = ":master")

@ianatol
Copy link
Member Author

ianatol commented Mar 18, 2022

This needed some minor fixes that will certainly mean the current PkgEval will fail, so rebasing, pushing, and starting a new one to run over night.

@ianatol
Copy link
Member Author

ianatol commented Mar 18, 2022

@nanosoldier runtests(ALL, vs = ":master")

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@ianatol
Copy link
Member Author

ianatol commented Mar 18, 2022

Ugh, yeah, some of these are definitely real. Looking into Evolutionary now, as its popped up a few times through our various runs

@Keno
Copy link
Member

Keno commented Mar 18, 2022

The one in Bonsai and ConstraintModels has a backtrace, so that should be easy to fix. But yeah, Evolutionary looks like a good one to look into.

@ianatol
Copy link
Member Author

ianatol commented Mar 18, 2022

Yeah, pretty sure those just need

@@ -597,7 +597,7 @@ function perform_lifting!(compact::IncrementalCompact,
         end
     end
 
-    the_leaf_val = (the_leaf::LiftedValue).x
+    the_leaf_val = isa(the_leaf, LiftedValue) ? the_leaf.x : nothing
     if !isa(the_leaf_val, SSAValue)
         all_same = false
     end

@Keno
Copy link
Member

Keno commented Mar 18, 2022

Yes, although it's interesting that this function is called with no leaves whatsoever. I'd need to look at the code again to think about whether that makes sense or not.

@ianatol
Copy link
Member Author

ianatol commented Mar 18, 2022

@nanosoldier runtests(["ARDESPOT", "BioStructures", "Bonsai", "ConstraintModels", "CopEnt", "CumulantsUpdates", "Diagonalizations", "Evolutionary", "FastMarching", "Fread", "GPUArrays", "Gettext", "HTTP", "MatrixPencils", "Reactive", "SimpleBufferStream", "SimpleFWA", "Simplices", "SuiteSparseMatrixCollection"], vs = ":master")

@ianatol
Copy link
Member Author

ianatol commented Mar 18, 2022

Bonsai, ConstraintModels, CumulantsUpdates, HTTP, and Gettext should be fixed by 4ee9414

Evolutionary seems to randomly fail on master as well: wildart/Evolutionary.jl#97

Still investigating others, but pretty hopeful they are stochastic considering none of them were in the previous run

@nanosoldier
Copy link
Collaborator

Your package evaluation job has completed - possible new issues were detected. A full report can be found here.

@ianatol ianatol force-pushed the kf/moresroa branch 2 times, most recently from 465af1c to 5626427 Compare March 19, 2022 00:53
@Keno Keno merged commit 7f29b70 into JuliaLang:master Mar 21, 2022
@ianatol ianatol deleted the kf/moresroa branch March 21, 2022 22:06
Keno added a commit that referenced this pull request Nov 28, 2023
This restriction has been in there since this code was added in #44557.
Unfortunately, I can't tell from the commit history why this restriction
was added. It's possible that it was trying to avoid putting things into
statement position that were not allowed in the phi block, but we have
cleaned that up since (#50308 and related), so let's see if this restriction
is still required, since I was seeing some suboptimial optimization results
because of this.
aviatesk pushed a commit that referenced this pull request Dec 1, 2023
This restriction has been in there since this code was added in #44557.
Unfortunately, I can't tell from the commit history why this restriction
was added. It's possible that it was trying to avoid putting things into
statement position that were not allowed in the phi block, but we have
cleaned that up since (#50308 and related), so let's see if this restriction
is still required, since I was seeing some suboptimial optimization results
because of this.
Keno added a commit that referenced this pull request Dec 7, 2023
…52338)

This restriction has been in there since this code was added in #44557.
Unfortunately, I can't tell from the commit history why this restriction
was added. It's possible that it was trying to avoid putting things into
statement position that were not allowed in the phi block, but we have
cleaned that up since (#50308 and related), so let's see if this
restriction is still required, since I was seeing some suboptimial
optimization results because of this.

---------

Co-authored-by: Shuhei Kadowaki <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants