Query Execution (Client/Server) - determination logic and perhaps notification/logging of the same - Hacks?

[Ankit Srivastava]

I was wondering if there is a way to figure out whether my Linq queries were evaluated at the client or the server side, may be via some flag, logs etc. in EF6 world?

Objective: We are in the process of migrating from EF6 to EFCore 6 and since client side evaluation is disabled in EFCore_v6, hence we need to know what all queries will fail to be exported from EF6.

I came across the below codes in EF6 which seems to have some determining logic as to whether the expression is evaluable at the Client or the Server side but I am not sure if that's the only place:

At Funcletizer.cs

// <summary>
        // Walks the expression tree and replaces client-evaluatable expressions with constants
        // or QueryParameterExpressions.
        // </summary>
        private sealed class FuncletizingVisitor : EntityExpressionVisitor
        {
            private readonly Funcletizer _funcletizer;
            private readonly Func<Expression, bool> _isClientConstant;
            private readonly Func<Expression, bool> _isClientVariable;
            private readonly List<Func<bool>> _recompileRequiredDelegates = new List<Func<bool>>();

           ........

And also the below at Translator.cs:

  // <summary>
            // This method is used to determine whether client side evaluation should be done,
            // if the property can be evaluated in the store, it is not being evaluated on the client
            // </summary>
            internal static bool CanFuncletizePropertyInfo(PropertyInfo propertyInfo)
            {
                PropertyTranslator propertyTranslator;
                // In most cases, we only allow funcletization of properties that could not otherwise be
                // handled by the query pipeline. ICollection<>.Count is the one exception to the rule
                // (avoiding a breaking change)
                return GenericICollectionTranslator.TryGetPropertyTranslator(propertyInfo, out propertyTranslator) ||
                       !TryGetTranslator(propertyInfo, out propertyTranslator);
            }

Could you please guide me to something where I can get some indication of whether the LINQ query that I am executing is either getting client evaluated or server evaluated or both (as in some part at client and other at the server end)?

Further technical details

EF version: EF 6.4.0
Database Provider: EntityFramework.SqlServer
Operating system: Windows 10
IDE: Visual Studio 2017

[Ankit Srivastava]

@ajcvickers @ErikEJ @AndriySvyryd Could anyone of you please share some insight on this?

[Erik Ejlskov Jensen]

Port your queries and run your integration test suite against a real database.

[Ankit Srivastava]

Port your queries and run your integration test suite against a real database.

Not sure what you meant with “Port your queries”, sorry?

I am looking for a solution with EF6 itself and Not with EF Core, if you were suggesting to use EF Core and run the queries there.

@ErikEJ any suggestions?

[Ankit Srivastava]

@ErikEJ @ajcvickers @AndriySvyryd @smitpatel Basically, all your customers who have been using EF6 and want to migrate to EF Core (v3 onwards, preferably v6 or v7) need some tool which can provide information about Query's execution behavior (what part got evaluated at the Client side and what on the server side).

A very similar and useful tool was there in the Telerik's Data Access

Could we not have such tool available for the customers who are migrating to EF Core world? Or at least give us some insights, so that we can build something very similar (either in EF6 (preferably) or EF Core v6)?

@powermetal63 Do you have any ideas on the above?

[Smit Patel]

tool which can provide information about Query's execution behavior (what part got evaluated at the Client side and what on the server side)

Can you please refer to this specific ask recurring in user issues?
Most our users who are using EF6 wants to migrate to EF Core and they want the queries which are running in EF6 runs in EF Core without throwing exception. In most cases, users are not even expecting to run the same SQL as long as it generates correct result. Given that EF6 also didn't allow client eval other than top level projection and EF Core follows same principle, there is not much gap in terms of client evaluation. There are few operators combination like GroupBy/GroupJoin which does require additional client evaluation which is not really translatable to database SQL directly and which EF6 performed and EFCore lacking. We have plan to address some of these in dotnet/efcore#24106

Irrespective of migration from EF6 to EF Core, a tool to identify the server/client eval part separately in a query may be interesting but not much useful to end customers as long as what EF does under the hood is correct performant results.

[Ankit Srivastava]

@smitpatel This thread and other queries around migration (EF6 to EF Core) are there only since there is a lack of clarity as to what will still be failing in newer version of EF Core. The statement "We have plan to address some of these in dotnet/efcore#24106" solves only half of the problem statement but does not shed light on what all things will still continue to fail (as against EF6).

1.) If by any chance we have a list of LINQ queries that used to work in EF6 and will not work in EF Core (even after v7 release) then we can actually try and focus only on those LINQs and try to find some work-around so that our Product's upgrade does not get hamper

2.) I agree that client side eval principle being similar in EF6 and EF Core, but since it is been disabled in EF Core (v3 onwards) we need to have a mechanism (code-based not by integration test suits) to determine the failing LINQ scenarios

Hence, if you could guide us on code base (EF6/EF Core) to areas where we could find such scenarios and flag them (by making changes and creating custom EF6/EFCore code base only for our use case) - that will make our migration full proof.

That is the only thing we are trying to figure out and asking you guys around.

[Ankit Srivastava]

Given that EF6 also didn't allow client eval other than top level projection and EF Core follows same principle, there is not much gap in terms of client evaluation.

@smitpatel I tried a simple sample from here msdn

var blogs = context.Blogs .OrderByDescending(blog => blog.Rating) .Select( blog => new { Id = blog.BlogId, Url = StandardizeUrl(blog.Url) }) .ToList();

public static string StandardizeUrl(string url) { url = url.ToLower(); if (!url.StartsWith("http://")) { url = string.Concat("http://", url); } return url; }

The above runs perfectly fine with EF Core v7.0.0-preview.2.22153.1 but throws System.NotSupportedException with EF6 v6.4.4

Exception:
LINQ to Entities does not recognize the method 'System.String StandardizeUrl(System.String)' method, and this method cannot be translated into a store expression.

Stack Trace:

at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.MethodCallTranslator.DefaultTranslator.Translate(ExpressionConverter parent, MethodCallExpression call) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.MethodCallTranslator.TypedTranslate(ExpressionConverter parent, MethodCallExpression linq) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TypedTranslator1.Translate(ExpressionConverter parent, Expression linq)
at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateExpression(Expression linq)
at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.NewTranslator.TypedTranslate(ExpressionConverter parent, NewExpression linq)
at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TypedTranslator1.Translate(ExpressionConverter parent, Expression linq) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateExpression(Expression linq) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateLambda(LambdaExpression lambda, DbExpression input) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateLambda(LambdaExpression lambda, DbExpression input, DbExpressionBinding& binding) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.MethodCallTranslator.OneLambdaTranslator.Translate(ExpressionConverter parent, MethodCallExpression call, DbExpression& source, DbExpressionBinding& sourceBinding, DbExpression& lambda) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.MethodCallTranslator.SelectTranslator.Translate(ExpressionConverter parent, MethodCallExpression call) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.MethodCallTranslator.SequenceMethodTranslator.Translate(ExpressionConverter parent, MethodCallExpression call, SequenceMethod sequenceMethod) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.MethodCallTranslator.TypedTranslate(ExpressionConverter parent, MethodCallExpression linq) at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TypedTranslator1.Translate(ExpressionConverter parent, Expression linq)
at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateExpression(Expression linq)
at System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.Convert()
at System.Data.Entity.Core.Objects.ELinq.ELinqQueryState.GetExecutionPlan(Nullable1 forMergeOption) at System.Data.Entity.Core.Objects.ObjectQuery1.<>c__DisplayClass41_0.b__1()
at System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[T](Func1 func, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction, Boolean releaseConnectionOnSuccess) at System.Data.Entity.Core.Objects.ObjectQuery1.<>c__DisplayClass41_0.b__0()
at System.Data.Entity.Infrastructure.DbExecutionStrategy.Execute[TResult](Func1 operation) at Lw.Sys.Repository.CustomDbExecutionStrategy.System.Data.Entity.Infrastructure.IDbExecutionStrategy.Execute[TResult](Func1 operation) in D:\Workspace\LW5\Framework4.0\Dev\Lw.System\Repository\CustomDbExecutionStrategy.cs:line 68
at System.Data.Entity.Core.Objects.ObjectQuery1.GetResults(Nullable1 forMergeOption)
at System.Data.Entity.Core.Objects.ObjectQuery1.<System.Collections.Generic.IEnumerable<T>.GetEnumerator>b__31_0() at System.Data.Entity.Internal.LazyEnumerator1.MoveNext()
at System.Collections.Generic.List1..ctor(IEnumerable1 collection)
at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at Lw.Sys.Test.SelectorTest.RunAllSelectors() in D:\Workspace\LW5\Framework4.0\Dev\Lw.System.Test\SelectorTests.cs:line 44

@ErikEJ @ajcvickers Going by your statement the above query is an example of "Top-Level Projection" which should have allowed client-side evaluation but fails in EF6, why?

Expecting a sooner response this time, thank you!

[Ankit Srivastava]

@smitpatel Could you please respond on the above query?

[Smit Patel]

I don't think anything has changed fundamentally in this issue from what I have responded last time here.

• There are queries which work in EF6 but doesn't work in EF Core. We plan to address most of them in EF Core 7. There may still be some queries which wouldn't work due to shear complexity and cost of it compared to value it provides but we will communicate that customer through better exception message. There won't be a documentation regarding it in a way since if a pattern is common enough to be documented then it should rather be supported.
• There are queries which fails in EF6 but works in EF Core. We don't plan to change that. The principal of client eval in top level projection has remained same in EF6 and EF Core. The reason of differing behavior, it is a principal not a code. Both the products tries to adhere to the principal as much as it can but doesn't necessarily follows it always. The difference happens due to architecture and design limitations of code. Further LINQ expression tree and/or SQL tree can be optimized in various way to achieve something which may look like client eval in intermediate stage but ends up not required it till top level projection. Whatever limitation EF6 had where it deviated from principal, neither we have time to do it nor there is any value in doing so. If EF Core is doing better than EF6 then it is already a plus point as long term that is what we recommend to our customers.

As for specific ask for a tool to tell if query will evaluate on client or server without actually writing such query, we cannot provide such tool. Not only there is not frequent ask for it (most customers wants that their query runs when migrating rather than caring about where does it evaluate what), from the infinite problem space of LINQ query world, I believe it is impossible to arrive at a tool which can accurately identify such behavior. Unless there is compelling reason to implement such tool and actually a logical solution which shows that such a tool can work correctly in most case (even if not accurately in 100% cases), we can look into it.