Compare commits

...

20 Commits

Author SHA1 Message Date
Richard Biener
271072ca25 Use NSDMI for predicate class
Use NSDMI for predicate class
2022-05-24 13:47:48 +02:00
Richard Biener
da159b3057 Another bogus range from .invert so don't use it.
From _1 < _2 we conclude [-INF, +INF - 1] and invert it as [+INF, +INF] -
oops.  Compute the other edge range explicitely.
2022-05-18 14:23:24 +02:00
Richard Biener
8d64734864 Properly only consider reachable BBs predicates 2022-05-18 13:59:26 +02:00
Richard Biener
399a970a9e Make sure to cost bb[0], work around unreachable predicates
This applies a workaround to avoid unswitching on predicates from
BBs that are considered unreachable and to that effect makes sure
to always cost bb[0] which we always put on the worklist as reachable.
2022-05-18 11:57:10 +02:00
Richard Biener
74ccb99070 This fixes the execute FAIL of gfortran.fortran-torture/execute/forall_7.f90
Using global ranges and inverting them is not correct - in the
particular case we ended up with a global range from a if (a_1 = b_2)
predicate and used the inversion on the false edge.  The change
refactors the predicate CTORs again, using range-ops to create a bare
range from the comparison in the GIMPLE_COND only and for switches
resorts to computing the ranges manually by unioning the case ranges.

The ranger query in the simplification code should probably change
as well to use a path_range_query where we should seed the path
range oracle with the predicate stack of the loop version we want
to simplify.
2022-05-18 10:59:23 +02:00
Richard Biener
ef6f1e1356 Avoid ICE when removing a case sharing the edge with the default case 2022-05-18 09:18:24 +02:00
Richard Biener
1990086f57 Merge remote-tracking branch 'origin/master' into unswitch
Bootstraps OK, has

FAIL: gfortran.fortran-torture/execute/forall_7.f90 execution,  -O3 -g

and ICEs 511.povray_r, 526.blender_r, 538.imagick_r with -Ofast znver2

splines.cpp:687:5: internal compiler error: in tree_unswitch_single_loop, at tree-ssa-loop-unswitch.cc:902

blender/source/blender/modifiers/intern/MOD_weightvg_util.c:120:6: internal compiler error: Segmentation fault

magick/composite.c:1631:32: internal compiler error: in tree_unswitch_single_loop, at tree-ssa-loop-unswitch.cc:902

the 902 is the size estimation assert recently added.
2022-05-17 19:10:08 +02:00
Richard Biener
3a3e129ce0 Really use first found predicate. Adjust limit for loop-unswitch-16.c 2022-05-17 17:39:22 +02:00
Richard Biener
0cc1e77123 fix ->aux clearing of duplicate_loop_body_to_header_edge 2022-05-17 17:38:51 +02:00
Richard Biener
88701062f3 Fix recursive unswitching regression
This fixes the recursive unswitching regression by introducing
the ability to track 'handled' as bitmap separately.  It also
fixes the budgeting to properly subtract the original loop size
(the param should probably be changed to a capped scale to not
allow arbitrary relative growth of small loops when there are a
lot of them?)
2022-05-17 16:03:59 +02:00
Richard Biener
ce7462de86 More cleanups
This performs more API cleanups, adds comments and fixes minor
issues.  It adds gcc.dg/loop-unswitch-16.c noting a regression
from the previous implementation.
2022-05-17 14:35:44 +02:00
Richard Biener
04951a4791 random fixes
restore unswitching on a < b with non-constant RHS.  Avoid quadratic
behavior when building predicates from switch stmts.
2022-05-17 11:56:48 +02:00
Richard Biener
68db341795 Avoid jump threading to defeat tests 2022-05-06 10:37:26 +02:00
Richard Biener
3281d212cb Merge remote-tracking branch 'origin/master' into unswitch 2022-05-05 14:51:46 +02:00
Richard Biener
6a11fbc73f move testcase that will conflict with one on trunk 2022-05-05 14:51:28 +02:00
Martin Liska
149d1e4ec9 Add top-level comment. 2022-01-13 17:02:38 +01:00
Martin Liska
1fb345c4ed Skip checking for ignored edges in find_unswitching_predicates_for_bb.
The discovery happens before propagation.
2022-01-07 10:49:06 +01:00
Martin Liska
694b57cbf4 Update scanned pattern and huge switches handling. 2022-01-07 09:56:34 +01:00
Martin Liska
401de0fee5 Apply reviewer's comments. 2022-01-06 17:30:08 +01:00
Martin Liska
1fdb1e9a24 Loop unswitching: refactoring & support for gswitch
gcc/ChangeLog:

	* dbgcnt.def (DEBUG_COUNTER): Add loop_unswitch counter.
	* tree-cfg.c (gimple_lv_add_condition_to_bb): Support not
	gimplified expressions.
	* tree-ssa-loop-unswitch.c (struct unswitch_predicate): New.
	(tree_unswitch_single_loop): Rework the function using
	pre-computed predicates.
	(tree_may_unswitch_on): Rename to ...
	(find_unswitching_predicates_for_bb): ... this.
	(clean_up_after_unswitching): New.
	(get_predicates_for_bb): Likewise.
	(set_predicates_for_bb): Likewise.
	(init_loop_unswitch_info): Likewise.
	(tree_ssa_unswitch_loops): Prepare stuff before calling
	tree_unswitch_single_loop.
	(merge_last): New.
	(add_predicate_to_path): Likewise.
	(find_range_for_lhs): Likewise.
	(simplify_using_entry_checks): Rename to ...
	(evaluate_control_stmt_using_entry_checks): ... this.
	(simplify_loop_version): Rework.
	(evaluate_insns): New.
	(evaluate_loop_insns_for_predicate): Likewise.
	(tree_unswitch_loop): Remove an assert.

gcc/testsuite/ChangeLog:

	* gcc.dg/loop-unswitch-10.c: New test.
	* gcc.dg/loop-unswitch-11.c: New test.
	* gcc.dg/loop-unswitch-12.c: New test.
	* gcc.dg/loop-unswitch-13.c: New test.
	* gcc.dg/loop-unswitch-6.c: New test.
	* gcc.dg/loop-unswitch-7.c: New test.
	* gcc.dg/loop-unswitch-8.c: New test.
	* gcc.dg/loop-unswitch-9.c: New test.
2022-01-06 14:02:21 +01:00
18 changed files with 1274 additions and 289 deletions

View File

@@ -187,6 +187,7 @@ DEBUG_COUNTER (ira_move)
DEBUG_COUNTER (ivopts_loop)
DEBUG_COUNTER (lim)
DEBUG_COUNTER (local_alloc_for_sched)
DEBUG_COUNTER (loop_unswitch)
DEBUG_COUNTER (match)
DEBUG_COUNTER (merged_ipa_icf)
DEBUG_COUNTER (phiopt_edge_range)

View File

@@ -14180,9 +14180,6 @@ The maximum depth of a loop nest suitable for complete peeling.
@item max-unswitch-insns
The maximum number of insns of an unswitched loop.
@item max-unswitch-level
The maximum number of branches unswitched in a single loop.
@item lim-expensive
The minimum cost of an expensive expression in the loop invariant motion.

View File

@@ -745,10 +745,6 @@ The maximum number of instructions to consider to unroll in a loop.
Common Joined UInteger Var(param_max_unswitch_insns) Init(50) Param Optimization
The maximum number of insns of an unswitched loop.
-param=max-unswitch-level=
Common Joined UInteger Var(param_max_unswitch_level) Init(3) Param Optimization
The maximum number of unswitchings in a single loop.
-param=max-variable-expansions-in-unroller=
Common Joined UInteger Var(param_max_variable_expansions) Init(1) Param Optimization
If -fvariable-expansion-in-unroller is used, the maximum number of times that an individual variable will be expanded during loop unrolling.

View File

@@ -0,0 +1,56 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized" } */
int
__attribute__((noipa))
foo(double *a, double *b, double *c, double *d, double *r, int size, int order)
{
for (int i = 0; i < size; i++)
{
double tmp, tmp2;
switch(order)
{
case 0:
tmp = -8 * a[i];
tmp2 = 2 * b[i];
break;
case 1:
tmp = 3 * a[i] - 2 * b[i];
tmp2 = 5 * b[i] - 2 * c[i];
break;
case 2:
tmp = 9 * a[i] + 2 * b[i] + c[i];
tmp2 = 4 * b[i] + 2 * c[i] + 8 * d[i];
break;
case 3:
tmp = 3 * a[i] + 2 * b[i] - c[i];
tmp2 = b[i] - 2 * c[i] + 8 * d[i];
break;
defaut:
__builtin_unreachable ();
}
double x = 3 * tmp + d[i] + tmp;
double y = 3.4f * tmp + d[i] + tmp2;
r[i] = x + y;
}
return 0;
}
#define N 16 * 1024
double aa[N], bb[N], cc[N], dd[N], rr[N];
int main()
{
for (int i = 0; i < 100 * 1000; i++)
foo (aa, bb, cc, dd, rr, N, i % 4);
}
/* Test that we actually unswitched something. */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 0" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 1" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 2" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 3" 1 "unswitch" } } */

View File

@@ -0,0 +1,45 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized" } */
int
foo(double *a, double *b, double *c, double *d, double *r, int size, int order)
{
for (int i = 0; i < size; i++)
{
double tmp, tmp2;
switch(order)
{
case 5 ... 6:
case 9:
tmp = -8 * a[i];
tmp2 = 2 * b[i];
break;
case 11:
tmp = 3 * a[i] - 2 * b[i];
tmp2 = 5 * b[i] - 2 * c[i];
break;
case 22:
tmp = 9 * a[i] + 2 * b[i] + c[i];
tmp2 = 4 * b[i] + 2 * c[i] + 8 * d[i];
break;
case 33:
tmp = 3 * a[i] + 2 * b[i] - c[i];
tmp2 = b[i] - 2 * c[i] + 8 * d[i];
break;
defaut:
__builtin_unreachable ();
}
double x = 3 * tmp + d[i] + tmp;
double y = 3.4f * tmp + d[i] + tmp2;
r[i] = x + y;
}
return 0;
}
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* \\+ 4294967291.*order.* == 9" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 1" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 2" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 3" 1 "unswitch" } } */

View File

@@ -0,0 +1,28 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized" } */
int
foo(double *a, double *b, double *c, double *d, double *r, int size, int order)
{
for (int i = 0; i < size; i++)
{
double tmp;
if (order == 1)
tmp = -8 * a[i];
else
tmp = -4 * b[i];
double x = 3 * tmp + d[i] + tmp;
if (order == 1)
x += 2;
double y = 3.4f * tmp + d[i];
r[i] = x + y;
}
return 0;
}
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 1" 1 "unswitch" } } */

View File

@@ -0,0 +1,35 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fno-thread-jumps -fdump-tree-unswitch-optimized" } */
int
foo(double *a, double *b, double *c, double *d, double *r, int size, unsigned order)
{
for (int i = 0; i < size; i++)
{
double tmp;
switch (order)
{
case 0 ... 4:
tmp = -8 * a[i];
break;
default:
tmp = -4 * b[i];
break;
}
double x = 3 * tmp + d[i] + tmp;
/* This and the case 0 ... 4 condition should only be unswitched once
since they are mutually excluded. */
if (order >= 5)
x += 2;
double y = 3.4f * tmp + d[i];
r[i] = x + y;
}
return 0;
}
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition" 1 "unswitch" } } */

View File

@@ -0,0 +1,60 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized --param=max-unswitch-insns=1000" } */
int
__attribute__((noipa))
foo(double *a, double *b, double *c, double *d, double *r, int size, int order)
{
for (int i = 0; i < size; i++)
{
double tmp, tmp2;
if (order <= 0)
tmp = 123;
switch(order)
{
case 0:
tmp += -8 * a[i];
tmp2 = 2 * b[i];
break;
case 1:
tmp = 3 * a[i] - 2 * b[i];
tmp2 = 5 * b[i] - 2 * c[i];
break;
case 2:
tmp = 9 * a[i] + 2 * b[i] + c[i];
tmp2 = 4 * b[i] + 2 * c[i] + 8 * d[i];
break;
case 3:
tmp = 3 * a[i] + 2 * b[i] - c[i];
tmp2 = b[i] - 2 * c[i] + 8 * d[i];
break;
defaut:
__builtin_unreachable ();
}
double x = 3 * tmp + d[i] + tmp;
double y = 3.4f * tmp + d[i] + tmp2;
r[i] = x + y;
}
return 0;
}
#define N 16 * 1024
double aa[N], bb[N], cc[N], dd[N], rr[N];
int main()
{
for (int i = 0; i < 100 * 1000; i++)
foo (aa, bb, cc, dd, rr, N, i % 4);
}
/* Test that we actually unswitched something. */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* <= 0" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 0" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 1" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 2" 1 "unswitch" } } */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 3" 1 "unswitch" } } */

View File

@@ -0,0 +1,15 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized" } */
void bar();
void baz();
void foo (int a, int b, int n)
{
for (int i = 0; i < n; ++i)
if (a < b)
bar ();
else
baz ();
}
/* { dg-final { scan-tree-dump "Unswitching loop on condition:" "unswitch" } } */

View File

@@ -0,0 +1,22 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized --param max-unswitch-insns=100" } */
void bar (int);
void foo (int a, int b, int c, int n)
{
for (int i = 0; i < n; ++i)
{
if (a > 5)
bar (1);
if (b < 10)
bar (2);
if (c != 5)
bar (3);
}
}
/* Verify we can unswitch all permutations of the predicates. */
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition" 7 "unswitch" } } */
/* { dg-final { scan-tree-dump "Unswitching loop on condition: a" "unswitch" } } */
/* { dg-final { scan-tree-dump "Unswitching loop on condition: b" "unswitch" } } */
/* { dg-final { scan-tree-dump "Unswitching loop on condition: c" "unswitch" } } */

View File

@@ -0,0 +1,24 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized" } */
int foo (int a)
{
do
{
if (a == 1)
return 0;
switch (a)
{
case 1:
return 5;
case 2:
return 7;
case 3:
return 11;
default:;
}
}
while (1);
}
/* { dg-final { scan-tree-dump-times "Unswitching loop" 3 "unswitch" } } */

View File

@@ -0,0 +1,28 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fno-thread-jumps -fdump-tree-unswitch-optimized" } */
int
foo(double *a, double *b, double *c, double *d, double *r, int size, float order)
{
for (int i = 0; i < size; i++)
{
double tmp;
if (order == 1.f)
tmp = -8 * a[i];
else
tmp = -4 * b[i];
double x = 3 * tmp + d[i] + tmp;
if (order == 1.f)
x += 2;
double y = 3.4f * tmp + d[i];
r[i] = x + y;
}
return 0;
}
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order.* == 1.0e" 1 "unswitch" } } */

View File

@@ -0,0 +1,31 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized" } */
int
foo(double *a, double *b, double *c, double *d, double *r, int size, int order)
{
for (int i = 0; i < size; i++)
{
double tmp;
if (order < 3)
tmp = -8 * a[i];
else
tmp = -4 * b[i];
double x = 3 * tmp + d[i] + tmp;
if (5 > order)
x += 2;
if (order == 12345)
x *= 5;
double y = 3.4f * tmp + d[i];
r[i] = x + y;
}
return 0;
}
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order" 3 "unswitch" } } */

View File

@@ -0,0 +1,27 @@
/* { dg-do compile } */
/* { dg-options "-O2 -funswitch-loops -fdump-tree-unswitch-optimized" } */
int
foo(double *a, double *b, double *c, double *d, double *r, int size, int order)
{
for (int i = 0; i < size; i++)
{
double tmp;
if (order == 1)
tmp = -8 * a[i];
else
{
if (order == 2)
tmp = -4 * b[i];
else
tmp = a[i];
}
r[i] = 3.4f * tmp + d[i];
}
return 0;
}
/* { dg-final { scan-tree-dump-times "Unswitching loop on condition: order" 2 "unswitch" } } */

View File

@@ -0,0 +1,39 @@
/* { dg-do compile } */
/* { dg-additional-options "-funswitch-loops" } */
enum {
MOD_WVG_MASK_TEX_USE_INT,
MOD_WVG_MASK_TEX_USE_RED,
MOD_WVG_MASK_TEX_USE_BLUE,
MOD_WVG_MASK_TEX_USE_SAT,
MOD_WVG_MASK_TEX_USE_VAL,
MOD_WVG_MASK_TEX_USE_ALPHA
} foo_num;
float *foo_org_w;
int *foo_new_w;
float foo_fact;
int foo_tex_use_channel, foo_i, foo_texres_0;
void foo()
{
for (; foo_num;)
switch (foo_tex_use_channel) {
case MOD_WVG_MASK_TEX_USE_INT:
foo_org_w[foo_i] = foo_new_w[foo_i] * foo_texres_0;
break;
case MOD_WVG_MASK_TEX_USE_RED:
foo_org_w[foo_i] = 0;
case MOD_WVG_MASK_TEX_USE_BLUE:
foo_org_w[foo_i] = foo_fact + foo_org_w[foo_i];
break;
case MOD_WVG_MASK_TEX_USE_SAT:
foo_org_w[foo_i] = foo_fact;
break;
case MOD_WVG_MASK_TEX_USE_VAL:
foo_org_w[foo_i] = 0;
case MOD_WVG_MASK_TEX_USE_ALPHA:
foo_org_w[foo_i] = foo_fact + foo_org_w[foo_i];
break;
default:
foo_org_w[foo_i] = foo_new_w[foo_i] * foo_texres_0;
}
}

View File

@@ -0,0 +1,14 @@
/* { dg-do compile } */
/* { dg-additional-options "-funswitch-loops" } */
int Get_Spline_Val_sp_0, Get_Spline_Val_k;
double Get_Spline_Val_p, Get_Spline_Val_se_0_0_0;
double *Get_Spline_Val_v;
void Get_Spline_Val() {
int i;
for (;;)
if (i > Get_Spline_Val_sp_0)
Get_Spline_Val_k = Get_Spline_Val_se_0_0_0;
else if (Get_Spline_Val_sp_0 == 1)
Get_Spline_Val_v[Get_Spline_Val_k] = Get_Spline_Val_p;
}

View File

@@ -9038,11 +9038,16 @@ gimple_lv_add_condition_to_bb (basic_block first_head ATTRIBUTE_UNUSED,
edge e0;
/* Build new conditional expr */
gsi = gsi_last_bb (cond_bb);
cond_expr = force_gimple_operand_gsi_1 (&gsi, cond_expr,
is_gimple_condexpr_for_cond,
NULL_TREE, false,
GSI_CONTINUE_LINKING);
new_cond_expr = gimple_build_cond_from_tree (cond_expr,
NULL_TREE, NULL_TREE);
/* Add new cond in cond_bb. */
gsi = gsi_last_bb (cond_bb);
gsi_insert_after (&gsi, new_cond_expr, GSI_NEW_STMT);
/* Adjust edges appropriately to connect new head with first head

File diff suppressed because it is too large Load Diff