Giters

microsoft / TypeScript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

Home Page:https://www.typescriptlang.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Suggestion: option to include undefined in index signatures

OliverJAsh opened this issue · comments

commented

Update: fixed by --noUncheckedIndexedAccess in TypeScript 4.1

Update: for my latest proposal see comment #13778 (comment)

With strictNullChecks enabled, TypeScript does not include undefined in index signatures (e.g. on an object or array). This is a well known caveat and discussed in several issues, namely #9235, #13161, #12287, and #7140 (comment).

Example:

const xs: number[] = [1,2,3];
xs[100] // number, even with strictNullChecks

However, it appears from reading the above issues that many TypeScript users wish this wasn't the case. Granted, if index signatures did include undefined, code will likely require much more guarding, but—for some—this is an acceptable trade off for increased type safety.

Example of index signatures including undefined:

const xs: number[] = [1,2,3];
xs[100] // number | undefined

I would like to know whether this behaviour could be considered as an extra compiler option on top of strictNullChecks. This way, we are able to satisfy all groups of users: those who want strict null checks with or without undefined in their index signatures.

commented

With the exception of strictNullChecks, we do not have flags that change the type system behavior. flags usually enable/disable error reporting.
you can always have a custom version of the library that defines all indexers with | undefined. should work as expected.

commented

@mhegazy That's an interesting idea. Any guidance on how to override the type signatures for array/object?

commented

There isinterface Array<T> in lib.d.ts. I searched by the regexp \[\w+: (string|number)\] to find other indexing signatures as well.

commented

Interesting, so I tried this:

{
    // https://github.com/Microsoft/TypeScript/blob/1f92bacdc81e7ae6706ad8776121e1db986a8b27/lib/lib.d.ts#L1300
    declare global {
        interface Array<T> {
            [n: number]: T | undefined;
        }
    }

    const xs = [1,2,3]
    const x = xs[100]
    x // still number :-(
}

Any ideas?

commented

copy lib.d.ts locally, say lib.strict.d.ts, change the index signature to [n: number]: T | undefined;, include the file in your compilation. you should see the intended effect.

commented

Cool, thanks for that.

The issue with the suggested fix here is it requires forking and maintaining a separate lib file.

I wonder if this feature is demanded enough to warrant some sort of option out of the box.

On a side note, it's interesting that the type signature for the get method on ES6 collections (Map/Set) returns T | undefined when Array/Object index signatures do not.

commented

this is a conscious decision. it would be very annoying for this code to be an error:

var a = [];
for (var i =0; i< a.length; i++) {
    a[i]+=1; // a[i] is possibly undefined
}

and it would be unreasonable to ask every user to use !. or to write

var a = [];
for (var i =0; i< a.length; i++) {
    if (a[i]) {
        a[i]+=1; // a[i] is possibly undefined
    }
}

For map this is not the case generally.

Similarly for your types, you can specify | undefined on all your index signatures, and you will get the expected behavior. but for Array it is not reasonable. you are welcome to fork the library and make whatever changes you need to do, but we have no plans to change the declaration in the standard library at this point.

I do not think adding a flag to change the shape of a declaration is something we would do.

commented

@mhegazy but for arrays with holes a[i] is actually possibly undefined:

let a: number[] = []
a[0] = 0
a[5] =0
for (let i = 0; i < a.length; i++) {
  console.log(a[i])
}

Output is:

0
undefined
undefined
undefined
undefined
0
commented

We remain quite skeptical that anyone would get any benefit from this flag in practice. Maps and maplike things can already opt in to | undefined at their definition sites, and enforcing EULA-like behavior on array access doesn't seem like a win. We'd likely need to substantially improve CFA and type guards to make this palatable.

If someone wants to modify their lib.d.ts and fix all the downstream breaks in their own code and show what the overall diff looks like to show that this has some value proposition, we're open to that data. Alternatively if lots of people are really excited to use postfix ! more but don't yet have ample opportunities to do so, this flag would be an option.

commented

We remain quite skeptical that anyone would get any benefit from this flag in practice. Maps and maplike things can already opt in to | undefined at their definition sites

Isn't one of the goals of TypeScript to allow errors to be caught at "compile" time rather than rely on the user to remember/know to do something specific? This seems to go against that goal; requiring the user to do something in order to avoid crashes. The same could be said for many other features; they're not needed if the developer always does x. The goal of TypeScript is (presumably) to make the job easier and eliminate these things.

I came across this bug because I was enabling strictNullChecks on existing code and I already had a comparison so I got the error. If I'd been writing brand new code I probably wouldn't have realised the issue here (the type system was telling me I always always getting a value) and ended up with a runtime failure. Relying on TS developers to remember (or worse, even know) that they're supposed to be declaring all their maps with | undefined feels like TypeScript is failing to do what people actually want it for.

commented

We remain quite skeptical that anyone would get any benefit from this flag in practice. Maps and maplike things can already opt in to | undefined at their definition sites
Isn't one of the goals of TypeScript to allow errors to be caught at "compile" time rather than rely on the user to remember/know to do something specific?

Actually the goal is:

  1. Statically identify constructs that are likely to be errors.

What is being discussed here the likelyhood of an error (low in the opinion of the TypeScript team) and the common productive usability of the language. Some of the early change to CFA have been to be less alarmist or improve the CFA analysis to more intelligently determine these things.

I think the question from the TypeScript team is that instead of arguing the strictly correctness of it, to provide examples of where this sort of strictness, in common usage would actually identify an error that should be guarded against.

commented

I went into the reasoning a bit more at this comment #11238 (comment)

Think of the two types of keys in the world: Those which you know do have a corresponding property in some object (safe), those which you don't know to have a corresponding property in some object (dangerous).

You get the first kind of key, a "safe" key, by writing correct code like

for (let i = 0; i < arr.length; i++) {
  // arr[i] is T, not T | undefined

or

for (const k of Object.keys(obj)) {
  // obj[k] is T, not T | undefined

You get the second kind from key, the "dangerous" kind, from things like user inputs, or random JSON files from disk, or some list of keys which may be present but might not be.

So if you have a key of the dangerous kind and index by it, it'd be nice to have | undefined in here. But the proposal isn't "Treat dangerous keys as dangerous", it's "Treat all keys, even safe ones, as dangerous". And once you start treating safe keys as dangerous, life really sucks. You write code like

for (let i = 0; i < arr.length; i++) {
  console.log(arr[i].name);

and TypeScript is complaining at you that arr[i] might be undefined even though hey look I just @#%#ing tested for it. Now you get in the habit of writing code like this, and it feels stupid:

for (let i = 0; i < arr.length; i++) {
  // TypeScript makes me use ! with my arrays, sad.
  console.log(arr[i]!.name);

Or maybe you write code like this:

function doSomething(myObj: T, yourObj: T) {
  for (const k of Object.keys(myObj)) {
    console.log(yourObj[k].name);
  }
}

and TypeScript says "Hey, that index expression might be | undefined, so you dutifully fix it because you've seen this error 800 times already:

function doSomething(myObj: T, yourObj: T) {
  for (const k of Object.keys(myObj)) {
    console.log(yourObj[k]!.name); // Shut up TypeScript I know what I'm doing
  }
}

But you didn't fix the bug. You meant to write Object.keys(yourObj), or maybe myObj[k]. That's the worst kind of compiler error, because it's not actually helping you in any scenario - it's only applying the same ritual to every kind of expression, without regard for whether or not it was actually more dangerous than any other expression of the same form.

I think of the old "Are you sure you want to delete this file?" dialog. If that dialog appeared every time you tried to delete a file, you would very quickly learn to hit del y when you used to hit del, and your chances of not deleting something important reset to the pre-dialog baseline. If instead the dialog only appeared when you were deleting files when they weren't going to the recycling bin, now you have meaningful safety. But we have no idea (nor could we) whether your object keys are safe or not, so showing the "Are you sure you want to index that object?" dialog every time you do it isn't likely to find bugs at a better rate than not showing it all.

commented

Statically identify constructs that are likely to be errors.

Perhaps this needs to be amended to say "Statically identify constructs that are more likely than others to be errors." 😉. I'm reminded of when we get bugs that are essentially "I used * when I meant to use /, can you using make * a warning?"

commented

I understand, but index out of range is a real and common issue; forcing people to enumerate arrays in a way that they can't do this would not be a bad thing.

The fix with ! I actually dislike too - what if someone comes along and makes a change such that the assumption is now invalid? You're back to square one (a potential runtime failure for something that the compiler should catch). There should be safe ways of enumerating arrays that do not rely on either lying about the types or using ! (eg. can't you do something like array.forEach(i => console.log(i.name)?).

You already narrow types based on code so in theory couldn't you could spot patterns that are safe narrow the type to remove | undefined in those cases, giving best of both worlds? I'd argue that if you can't easily convey to the compiler that you're not accessing a valid element then maybe your guarantee is either invalid or could easily be accidentally be broken in future.

That said, I only use TS on one project and that will ultimately be migrated to Dart so it's unlikely to make any real difference to me. I'm just sad that the general quality of software is bad and there's an opportunity to help eliminate errors here that is seemingly being ignored for the sake of convenience. I'm sure the type system could be made solid and the common annoyances addressed in a way that doesn't introduce these holes.

Anyway, that's just my 2 cents.. I don't want to drag this out - I'm sure you understand where we're coming from and you're far better placed to make decisions on this than me :-)

commented

I think there are a few things to consider. There are a lot of patterns for iterating over arrays in common use that account for the number of elements. While it is a possible pattern to just randomly access indexes on arrays, in the wild that is a very uncommon pattern and is not likely to be a statical error. While there are modern ways to iterate, the most common would be something like:

for (let i = 0; i < a.length; i++) {
  const value = a[i];
}

If you assume spare arrays are uncommon (they are) it is of little help to have value be | undefined. If there is a common pattern, in the wild, where this is risky (and likely an error) then I think the TypeScript would listen to consider this, but the patterns that are in general use, having to again again against all values of an index access be undefined is clearly something that affects productivity and as pointed out, can be opted into if you are in a situation where it is potentially useful.

I think there has been conversation before about improving CFA so that there is a way to express the co-dependancy of values (e.g. Array.prototype.length relates to the index value) so that things like index out of bounds could be statically analysed. Obviously that is a significant piece of work, wrought with all sorts of edge cases and considerations I wouldn't like to fathom (though it is likely Anders wakes up in a cold sweat over some things like this).

So it becomes a trade off... Without CFA improvements, complicate 90% of code with red herrings to catch potentially 10% bad code. Otherwise it is investing in major CFA improvements, which might be wrought with their own consequences of stability and issues against again, finding what would be unsafe code.

There is only so much TypeScript can do to save us from ourselves.

commented

All this focus is on arrays and I agree it's less likely to be an issue there, but most of the original issues raised (like mine) were about maps where I don't think the common case is always-existing keys at all?

commented

All this focus is on arrays and I agree it's less likely to be an issue there, but most of the original issues raised (like mine) were about maps where I don't think the common case is always-existing keys at all?

If this is your type, add | undefined to the index signature. It is already an error to index into an type with no index signature under --noImplicitAny.
ES6 Map is already defined with get as get(key: K): V | undefined;.

commented

i rewrote all definitions of Arrays and Maps to make index signatures returning | undefined, never regreted since that, found a few bugs, it doesn't cause any discomfort because i work with arrays indirectly via a handmade lib that keeps checks for undefined or ! inside of it

would be great if TypeScript could control flow the checks like C# does (to eliminate index range checks to save some processor time), for example:

declare var values: number[];
for (let index = 0, length = values.length; index< length; index ++) {
   const value = value[index]; // always defined, because index is within array range and only controlled by it
}

(to those who uses sparse arrays - kill yourself with hot burning fire)

as for Object.keys, it takes a special type say allkeysof T to let the control flow analysis do safe narrowings

commented

I think this would be a good option to have, because right now we are essentially lying about the type of the indexing operation, and it can be easy to forget to add | undefined to my object types. I think adding ! in the cases where we know we want to ignore undefineds would be a nice way to deal with indexing operations when this option is enabled.

commented

There are (at least) two other problems with putting |undefined in your object type definitions:

  • it means you can assign undefined into these objects, which is definitely not intended
  • other operations, like Object.values (or _.values) will require you to handle undefined in the results
commented

tslint reports false-positive warning of constant condition, because typescript returns wrong type information (= lacking | undefined).
palantir/tslint#2944

commented

One of the regularly skipped errors with the absence of | undefined in the array indexing is this pattern when used in place of find:

const array = [ 1, 2, 3 ];
const firstFour = array.filter((x) => (x === 4))[0];
// if there is no `4` in the `array`,
// `firstFour` will be `undefined`, but TypeScript thinks `number` because of the indexer signature.
const array = [ 1, 2, 3 ];
const firstFour = array.find((x) => (x === 4));
// `firstFour` will be correctly typed as `number | undefined` because of the `find` signature.
commented

I would definitely use this flag. Yes, old for loops will be annoying to work with, but we have the ! operator to tell the compiler when we know it's defined:

for (let i = 0; i < arr.length; i++) {
  foo(arr[i]!)
}

Also, this problem is not a problem with the newer, way better for of loops and there is even a TSLint rule prefer-for-of that tells you to not use old-style for loops anymore.

Currently I feel like the type system is inconsistent for the developer. array.pop() requires an if check or a ! assertion, but accessing via [array.length - 1] does not. ES6 map.get() requires an if check or a ! assertion, but an object hash does not. @sompylasar's example is also good.

Another example is destructuring:

const specifier = 'Microsoft/TypeScript'
const [repo, revision] = specifier.split('@') // types of repo and revision are string
console.log('Repo: ' + repo)
console.log('Short rev: ' + revision.slice(0, 7)) // Error: Cannot call function 'slice' on undefined

I would have preferred if the compiler forced me to do this:

const specifier = 'Microsoft/TypeScript'
const [repo, revision] = specifier.split('@') // types of repo and revision are string | undefined
console.log('Repo: ', repo || 'no repo')
console.log('Short rev:', revision ? revision.slice(0, 7) : 'no revision')

These are actual bugs I've seen that could have been prevented by the compiler.

Imo this shouldn't belong into the typings files, but should rather be a type system mechanic - when accessing anything with an index signature, it can be undefined. If your logic ensured that it isn't, just use !. Otherwise add an if and you're good.

I think a lot of people would prefer the compiler to be strict with some needed assertions than to be loose with uncaught bugs.

commented

I'd really like to see this flag added. In my company's code base, array random access is the rare exception and for loops are code smells that we'd usually want to rewrite with higher-order functions.

commented

@pelotom what is your concern then (since it seems you mostly got yourself out of trouble)?

commented

@Aleksey-Bykov mostly object index signatures, which occur extensively in third-party libraries. I would like accessing a property on { [k: string]: A } to warn me that the result is possibly undefined. I only mentioned array indexing because it was brought up as a case for why the flag would be too annoying to work with.

commented

you know you can rewrite them exactly the way you want? (given a bit of extra work)

commented

Yes, I could rewrite everyone's typings for them, or I could switch on a compiler flag 😜

commented

keep playing captain O...: you can rewrite your lib.d.ts today and be a happy owner of more sound codebase or you can wait for the flag for the next N years

commented

@Aleksey-Bykov how can it be done by rewriting lib.d.ts?

commented 
declare type Keyed<T> = { [key: string]: T | undefined; }
commented

then in the Array defintion in lib.es2015.core.d.ts, replace

[n: number]: T;

with

[n: number]: T | undefined;
commented

@Aleksey-Bykov maybe you missed the part where I said I don't care about arrays. I care about where third party libraries have declared something to be of type { [k: string]: T }, and I want accessing such an object to return something possibly undefined. There's no way to accomplish that by simply editing lib.d.ts; it requires changing the signatures of the library in question.

commented

do you have control over 3rd party definition files? if so you can fix them

commented

And now we're back to

Yes, I could rewrite everyone's typings for them, or I could switch on a compiler flag 😜

Time is a flat circle.

commented

don't be silly, you don't use "everyone's typings" do you? it's literally a day of work max for a typical project, been there done it

commented

Yes, I have to edit others' typings all the time and I'd like to do it less.

commented

and you will in N years, maybe, for now you can suffer or man up

commented

Thanks for your incredibly constructive input 👍

commented

constructive input for this is as follows, this issue needs to be closed, because:
a. either the decision on whether [x] can be undefined or not is left to the developers by

  • letting them keep it all in their heads as they always did before
  • or by altering lib.d.ts and 3rd-party.d.ts as was suggested

b. or it takes special syntax / types / flow analysis / N years to mitigate something that can be easily done by hands in #a

commented

The issue is a proposal for (b), except no new syntax is being proposed, it's just a compiler flag.

What it comes down to is that the type { [x: string]: {} } is almost always a lie; barring the use of Proxy, there's no object which can have an infinite number of properties, much less every possible string. The proposal is to have a compiler flag which recognizes this. It may be that it's too hard to implement this for what is gained; I'll leave that call to the implementors.

commented

the point is that neither

  • T | undefined
  • nor T

is right for the general case

in order to make it right for the general case you need to encode the information about the prerense of values into the types of their containers which calls for a dependent type system ... which by itself isn't a bad thing to have :) but might be as complex as all current typescript type system done to this day, for the sake of ... saving you some edits?

commented

T | undefined is correct for the general case, for reasons I just gave. Gonna ignore your nonsensical ramblings about dependent types, have a nice day.

commented

you can ignore me as much as you want but T | undefined is an overshoot for

declare var items: number[];
for (var index = 0; index < items.length; index ++) {
   void items[index];
}
commented

I'd rather have T | undefined there by default and tell the compiler that index is a numeric index range of items thus doesn't get out if bounds when applied to items; in the simple cases such as a set of frequently used for/while loop shapes, the compiler could infer that automatically; in complex cases, sorry, there can be undefineds. And yes, value-based types would be a good fit here; literal string types are so useful, why not have literal boolean and number and range/set-of-ranges types? As far as TypeScript goes, it tries to cover everything that can be expressed with JavaScript (in contrast to, for example, Elm which limits that).

commented

it's literally a day of work max for a typical project, been there done it

@Aleksey-Bykov, curious what was your experience after that change? how often do you have to use !? and how often do you find the compiler flagging actual bugs?

commented

@mhegazy honestly i didn't notice much difference moving from T to T | undefined, neither did i catch any bugs, i guess my problem is that i work with arrays via utility functions which keep ! in them, so literally there was no effect for the outside code:

https://github.com/aleksey-bykov/basic/blob/master/array.ts

commented

In which lib file can I find the index type definition for objects? I have located and updated Array from [n: number]: T to [n: number]: T | undefined. Now I would like to do the same thing for objects.

commented

there is no standard interface (like Array for arrays) for objects with the index signature, you need to look for exact definitions per each case in your code and fix them

commented

you need to look for exact definitions per each case in your code and fix them

How about a direct key lookup? E.g.

const xs = { foo: 'bar' }
xs['foo']

Is there any way to enforce T | undefined instead of T here? Currently I use these helpers in my codebase everywhere, as type safe alternatives to index lookups on arrays and objects:

// TS doesn't return the correct type for array and object index signatures. It returns `T` instead
// of `T | undefined`. These helpers give us the correct type.
// https://github.com/Microsoft/TypeScript/issues/13778
export const getIndex = function<X> (index: number, xs: X[]): X | undefined {
  return xs[index];
};
export const getKeyInMap = function<X> (key: string, xs: { [key: string]: X }): X | undefined {
  return xs[key];
};

@mhegazy As I write this, I am fixing a bug in production on https://unsplash.com that could have been caught with stricter index signature types.

commented

i see, consider mapped type operator:

const xs = { foo: 'bar' };
type EachUndefined<T> = { [P in keyof T]: T[P] | undefined; }
const xu : EachUndefined<typeof xs> = xs;
xu.foo; // <-- string | undefined
commented

If a flag like --strictArrayIndex is not an option because flags are not designed to change the lib.d.ts files. Maybe you guys can release strict versions of lib.d.ts files like "lib": ['strict-es6']?

It could contain multiple improvements, not just strict array index. For example, Object.keys:

interface ObjectConstructor {
    // ...
    keys(o: {}): string[];
}

Could be:

interface ObjectConstructor {
    // ...
    keys<T>(o: T): (keyof T)[];
}
commented

Update from today's SBS: We yelled at each other for 30 minutes and nothing happened. 🤷‍♂️

commented

@RyanCavanaugh What's an SBS, out of curiosity?

commented

@radix "Suggestion Backlog Slog"

commented

thats intriguing, because the solution is obvious:
both Tand T | undefined are wrong, the only right way is to make the index variable aware of the capacity of its container, either by picking from a set or by enclosing it in a known numeric range

commented

@RyanCavanaugh i ve been thinking abou this, it looks like the following trick would cover 87% of all cases:

  • values[index] gives T if index declared in for (HERE; ...)
  • values[somethingElse] gives T | undefined for all variables declared outside of for
commented

@Aleksey-Bykov we discussed something even smarter - that there could be an actual type guard mechanism for "arr[index] was tested by index < arr.length. But that wouldn't help the case where you pop'd in the middle a loop, or passed your array to a function that removed elements from it. It really doesn't seem like people are looking for a mechanism that prevents OOB errors 82.9% of the time -- after all, it's already the case that approximately that fraction of array-indexing code is correct anyway. Adding ceremony to correct code while not catching bugs in the incorrect cases is a bad outcome.

commented

not to imply that Aleksey would ever mutate an array

commented

I ported an app from Elm to Typescript recently and indexing operations being incorrectly typed is probably one of the biggest sources of bugs I've run into, with all the strictest settings of TS enabled (also stuff like this being unbound).

commented
  • you cant track mutations to the container
  • you cant track index manipulations

with this said little can you guarantee, if so why would you even try if a typical use case is dumb iteration through all elements within < array.length

like i said usually

  • if there is an error, it is likely originates from somewhere in the clauses of the for statement: initialization, increment, stop condition and thus is not something you can verify, because it requires turing complete typechecking
  • outside of for clauses there is usually no place for error

so as long as index is declared somehow (nothing we can say about it with confidence) it is reasonably fair to believe that the index is correct withing the loop body

commented

But that wouldn't help the case where you pop'd in the middle a loop, or passed your array to a function that removed elements from it

This seems like a really weak argument against this. Those cases are already broken - using them as an excuse not to fix other cases makes no sense. Nobody here is asking for either of those cases to be handled correctly nor for it to be 100% correct. We just want accurate types in a really common case. There are many cases in TypeScript that aren't handled perfectly; if you're doing something weird, types might be wrong. In this case we're not doing anything weird and the types are wrong.

Adding ceremony to correct code

I'm curious to see a code example of where this is "adding ceremony to correct code". As I understand it, a basic for loop over an array is easy to detect/narrow. What's the real world code that isn't a simple for loop where this becomes inconvenient which is not potentially a bug? (either now, or could be from a trivial change in the future). I'm not saying there isn't any; I just can't picture it and haven't seen any examples (I've seen plenty of examples using for loops, but unless you're saying these can't be narrowed they seem irrelevant).

while not catching bugs

There have been examples of both working code that fails to compile because of this and code that throws at runtime because the types mislead the developer. Saying there is zero value in supporting this is nonsense.

commented

Why can't we just keep it simple and not add any magic behaviour around old-style for loops at all. You can always use ! to make things work. If your codebase is full of old-style for loops, don't use the flag. All modern codebases I've worked with use either forEach or for of to iterate arrays and those codebases would benefit from the additional type safety.

commented

we're not doing anything weird and the types are wrong.

This paragraph, to me, reads like a good reason to not have this feature. Accessing an array out-of-bounds is a weird thing to do; the types are only "wrong" if you're doing an uncommon thing (OOBing). The vast majority of code reading from an array does so in-bounds; it would be "wrong" to include undefined in those cases by this argument.

... working code that fails to compile

I'm not aware of any - can you point to them specifically?

Why can't we just keep it simple and not add any magic behaviour around old-style for loops at all. You can always use ! to make things work.

What's the useful difference between this and a TSLint rule that says "All array access expressions must have a trailing !" ?

commented

@RyanCavanaugh my assumption is that the TSLint rule would not be able to narrow types or use control-flow type analysis (e.g. wrapping the access in an if, throwing an exception, returning or continueing if it is not set, etc). It is also not just about array access expressions, it is also about destructuring. How would the implementation look like for the example in #13778 (comment)? To enforce the !, it would essentially have to create a table to track types of these variables as being possibly undefined. Which to me sounds like something that the type checker should do, not a linter.

commented

@RyanCavanaugh

This paragraph, to me, reads like a good reason to not have this feature. Accessing an array out-of-bounds is a weird thing to do; the types are only "wrong" if you're doing an uncommon thing (OOBing).

... working code that fails to compile

I'm not aware of any - can you point to them specifically?

In my case it wasn't an array, but it was closed as a dupe so I assume this issue is supposed to be covering these too. See #11186 for my original issue. I was parsing a file into a map and then checking them against undefined. IIRC I got the error on the comparison that they can't be undefined, even though they could (and were).

commented

It's always allowed to compare something to undefined

commented

which is a shame

const canWeDoIt = null === undefined; // yes we can!
commented

It's always allowed to compare something to undefined

It's been so long, I'm afraid I don't remember exactly what the error was. I definitely had an error for code that was working fine (without strictNullChecks) and it lead me to the testing that resulted in the case above.

If I get some time, I'll see if I can figure it out exactly what it was again. It was definitely related to this typing.

commented

Most of this discussion has been focused on arrays, but, in my experience, object access is where this feature would be most helpful. For example, this (simplified) example from my codebase looks reasonable and compiles fine, but is definitely a bug:

export type Chooser = (context?: Context) => number | string;
export interface Choices {
    [choice: number]: Struct;
    [choice: string]: Struct;
}

export const Branch = (chooser: Chooser, choices: Choices, context?: Context): Struct => {
    return choices[chooser(context)];  // Could be undefined
}
commented

Regarding objects and simply changing the signature to include | undefined, @types/node does that for process.env:

    export interface ProcessEnv {
        [key: string]: string | undefined;
    }

but it doesn't allow narrowing the type at all:

process.env.SOME_CONFIG && JSON.parse(process.env.SOME_CONFIG)

gives

[ts]
Argument of type 'string | undefined' is not assignable to parameter of type 'string'.
  Type 'undefined' is not assignable to type 'string'.
commented

@felixfbecker

I believe that is related to this bug: #17960

You can workaround it by assigning the env var to a variable, and then guarding that:

const foo = process.env.SOME_CONFIG
foo && JSON.parse(foo);
commented

Adding the | undefined in some node typings was done to allow augmentation of these interface for convenience with frequently used properties to get code completion. e.a.

interface ProcessEnv {
  foo?: string;
  bar?: string;
}

Without the | undefined in the index signature TSC complains with Property 'foo' of type 'string | undefined' is not assignable to string index type 'string'..

I know that this has a cost if you have to work with types having the extra | undefined and ones which don't have it.

Would be really nice if we could get rid of this.

commented

I'm trying to collect my thoughts on this issue.

Objects are a mixed bag in JavaScript. They can be used for two purposes:

  • dictionaries aka maps, where the keys are unknown
  • records, where the keys are known

For objects used as records, index signatures do not need to be used. Instead, known keys are defined on the interface type. Valid key lookups return the corresponding type, invalid key lookups will fallback to the index signature. If there is no index signature defined (which there shouldn't be for objects used as records), and noImplicitAny is enabled, this will error as desired.

For objects used as dictionaries (aka maps) and arrays, index signatures are used, and we can choose to include | undefined in the value type. For example, { [key: index]: string | undefined }. All keys lookups are valid (because keys are not known at compile time), and all keys return the same type (in this example, T | undefined).

Seeing as index signatures should only be used for the dictionary objects pattern and arrays, it is desired that TypeScript should enforce | undefined in the index signature value type: if the keys are not known, and key lookups possibly return undefined.

There are good examples of bugs that may appear invisible without this, such as Array.prototype.find returning undefined, or key lookups such as array[0] returning undefined. (Destructuring is just sugar syntax for key lookups.) It is possible to write functions like getKey to correct the return type, but we have to rely on discipline to enforce use of these functions across a codebase.

If I understand correctly, the issue then becomes about reflection over dictionary objects and arrays, such that when mapping over the keys of a dictionary object or array, key lookups are known to be valid i.e. will not return undefined. In this case, it would be undesirable for value types to include undefined. It may be possible to use control flow analysis to fix this.

Is this the outstanding issue, or do I misunderstand?

Which use cases and problems haven't I mentioned?

commented

There are (at least) two other problems with putting |undefined in your object type definitions:

  • it means you can assign undefined into these objects, which is definitely not intended
  • other operations, like Object.values (or _.values) will require you to handle undefined in the results

I think this is a very important point.

At the moment, I am experimenting with the following approach:

Define const safelyAccessProperty = <T, K extends keyof T>(object: T, key: K): T[K] | undefined => object[key];

Then access properties like safelyAccessProperty(myObject, myKey), instead of myObject[myKey].

commented

@plul Good catch. The discussion is currently focused on read operations, but the indexer type definition is in fact two-fold, and adding | undefined would allow writing undefined values.

The safelyAccessProperty function you are experimenting with (mentioned above as getKey by @OliverJAsh) requires discipline and/or a linter rule to forbid indexing operations on all arrays and objects.

This can be made scalable if the function is provided on all array and object instances (every type that provides indexer operations), like in C++ std::vector has .at() which throws an exception in runtime for OOB access, and an unchecked [] operator which in best case crashes with SEGFAULT on OOB access, in worst case corrupts memory.

I think the OOB access problem is not solvable in TypeScript/JavaScript at the type definition level alone, and requires language support to restrict potentially dangerous indexer operations if this strictness feature is enabled.

The two-fold nature of the indexer could be modeled as a property with get and set operations as functions, but that would be a breaking change for all existing indexer type definitions.

commented

One idea that seems like it could be promising: what if you could use ?: when defining an index signature to indicate that you expect values to be missing sometimes under normal use? It would act like | undefined mentioned above, but without the awkward downsides. It would need to disallow explicit undefined values, which I guess is a difference from usual ?:.

It would look like this:

type NewWay = {[key: string]?: string};
const n: NewWay = {};

// Has type string | undefined
n['foo']

// Has type Array<string>
Object.values(n)

// Doesn't work
n['foo'] = undefined;

// Works
delete n['foo'];

Compared to the previous approach of | undefined:

type OldWay = {[key: string]: string | undefined};
const o: OldWay = {};

// Has type string | undefined
o['foo']

// Has type Array<string | undefined>
Object.values(o)

// Works
o['foo'] = undefined;

// Works
delete o['foo'];
commented

I came here from rejecting adding | undefined in the DT PR above, as it would break all the existing users of that API - could this be better looked at as allowing the user to pick how fussy they want to be, rather than the library?

I'll note optional properties add the | undefined as well, and that has bitten me a few times - essentially TS doesn't distinguish between a missing property and a property set to undefined. I would just like { foo?: T, bar?: T } to be treated the same as { [name: 'foo' | 'bar']: T }, whichever way that goes (see also the process.env comments above)

Is TS against breaking the symmetry here on number and string indexers?

foo[bar] && foo[bar].baz() is a very common JS pattern, it feels clumsy when it's not supported by TS (in the sense of reminding you that you need to if you don't add | undefined, and warning when it's obviously not required if you do).

Regarding mutating arrays during iteration breaking the guard expression guarantee, that's possible with the other guards, too:

class Foo {
    foo: string | number = 123

    bar() {
        this.foo = 'bar'
    }

    broken() {
        if (typeof this.foo === 'number') {
            this.bar();
            this.foo.toLowerCase(); // right, but type error
            this.foo.toExponential(); // wrong, but typechecks
        }
    }
}

but I guess that's a lot less likely in real code that old loops mutating the iteratee.

commented

It's clear that there is a demand for this feature. I really hope TS team will find some solution. Not by just adding | undefined to indexer because it has it's own issues (mentioned already) but by more "clever" way (reading returns T|undefined, writing requires T, good compiler checking for for loop, etc. good proposition were also mentioned already.)

We are fine with runtime error when we mutate and work with arrays in non trivial, difficult to verify by compiler way. We just want error checking for most cases and are fine with using ! sometimes.

Having said that, if this stricter handling of arrays would be implemented, now with #24897 it would be possible to implement nice type narrowing when checking array length with constant. We could just narrow array to tuple with rest element.

let arr!: string[];
if (arr.length == 3) {
  //arr is of type [string, string, string]
}

if (arr.length > 3) {
  //arr is of type [string, string, string, string, ...string[]]
}

if (arr.length) {
  //arr is of type [string, ...string[]]
}

if (arr.length < 3) {
  //arr is of type [string?, string?, string?]
  if (arr.length > 0) {
    //arr is of type [string, string?, string?]
  }
}

It would be useful when you index by constant or destructure an array.

let someNumber = 55;
if (arr.length) {
  let el1 = arr[0]; //string
  let el2 = arr[1]; //string | undefined
  let el3 = arr[someNumber]; //string | undefined
}

if(arr.length >= 3){
    let [el1, el2, el3, el4] = arr;
    //el1, el2, el3 are string
    // el4 is string | undefined    
}

if (arr.length == 2){
    let [el1, el2, el3] = arr; //compiler error: "Tuple type '[string, string]' with length '2' cannot be assigned to tuple with length '3'.",
}

Other question is what would we do with big numbers like:

if(arr.length >= 99999){
    // arr is [string, string, ... , string, ...string[]]
}

We can't show the type of this huge tuple in IDE or compiler messages.

I guess we could have some syntax to represent "tuple of certain length with same type for all items". So for example the tuple of 1000 strings is string[10000] and the type of narrowed array from above example could be [...string[99999], ...string[]].

The other concern is if compiler infrastructure can support such big tuples now, and if not how hard would it be to do.

commented

Objects

I always want an index type of [key: string (or number, symbol)]: V | undefined, only sometimes I forget about the undefined case. Whenever a dev has to explicitely tell the compiler "trust me, this is the type of that thing for real" you know it's unsafe.
It makes very little sense to type Map.get properly (strictly) but somehow plain objects get a free pass.
Still, this is easily fixable in user land, so it's not too bad. I don't have a solution, at any rate.

Arrays

Perhaps I'm missing something but it seems the argument that "you almost never access an Array in an unsafe way" can go both ways, especially with a new compiler flag.

I tend to think more and more people follow these two best practices:

  • Use functional native methods or libraries to iterate or transform Arrays. No bracket access here.
  • Don't mutate Arrays in place

With this in mind, the only remaining and rare cases where you need low level bracket access logic would really benefit from type safety.

This one seems like a no brainer and I don't think copy-pasting the entire lib.d.ts locally is an acceptable workaround.

commented

When we explicitly index into an array/object, e.g. to get the first item of an array (array[0]), we want the result to include undefined.

This is possible by adding undefined to the index signature.

However, if I understand correctly, the issue with including undefined in the index signature is that, when mapping over the array or object, the values will include undefined even though they're known not to be undefined (otherwise we wouldn't be mapping over them).

Index types/signatures are used for both index lookups (e.g. array[0]) and mapping (e.g. for loops and Array.prototype.map), but we require different types for each of these cases.

This is not an issue with Map because Map.get is a function, and therefore its return type can be separate from the inner value type, unlike indexing into an array/object which is not a function, and therefore uses the index signature directly.

So, the question becomes: how can we satisfy both cases?

// Manually adding `undefined` to the index signature
declare const array: (number | undefined)[];

const first = array[0]; // number | undefined, as desired :-)
type IndexValue = typeof array[0]; // number | undefined, as desired! :-)

array.map(x => {
  x // number | undefined, not desired! :-(
})

Proposal

A compiler option which treats index lookups (e.g. array[0]) similar to how Set.get and Map.get are typed, by including undefined in the index value type (not the index signature itself). The actual index signature itself would not include undefined, so that mapping functions are not effected.

Example:

declare const array: number[];

// The compiler option would include `undefined` in the index value type
const first = array[0]; // number | undefined, as desired :-)
type IndexValue = typeof array[0]; // number | undefined, as desired :-)

array.map(x => {
  x // number, as desired :-)
})

This however won't solve the case of looping over array/objects using a for loop, as that technique uses index lookups.

for (let i = 0; i < array.length; i++) {
  const x = array[i];
  x; // number | undefined, not desired! :-(
}

For me and I suspect many others, this is acceptable because for loops are not used, instead preferring to use functional style, e.g. Array.prototype.map. If we did have to use them, we would be happy using the compiler option suggested here along with non-null assertion operators.

for (let i = 0; i < array.length; i++) {
  const x = array[i]!;
  x; // number, as desired :-)
}

We could also provide a way to opt-in or opt-out, e.g. with some syntax to decorate the index signature (please forgive the ambiguous syntax I came up with for the example). This syntax would just be a way of signalling which behaviour we want for index lookups.

Opt-out (compiler option enables by default, opt-out where needed):

declare const array: { [index: number]!!: string };

declare const dictionary: { [index: string]!!: string }

Opt-in (no compiler option, just opt-in where needed):

declare const array: { [index: string]!!: string };

declare const dictionary: { [index: string]??: string }
commented

I haven't read up on this issue or the pros and cons, various proposals, etc. (just found it in Google after being repeatedly surprised that array/object access isn't handled consistently with strict null checks), but I have a related suggestion: an option to make array type inference as strict as possible unless specifically overridden.

For example:

const balls = [1, 2 ,3];

By default, balls would be treated as [number, number, number]. This could be overridden by writing:

const balls: number[] = [1, 2 ,3];

Further, tuple element access would be handled consistently with strict null checks. It's surprising to me that in the following example n is currently inferred as number even with strict null checks enabled.

const balls: [number, number, number] = [1, 2 ,3];
const n = balls[100];

I would also expect array mutation methods such as .push to not exist in the tuple type definition, since such methods change the run-time type to be inconsistent with the compile-time type.

commented

Nice! Well that's interesting timing. I'd had the release announcement open, but hadn't read through it yet; only came here after running into a situation where I needed to do some weird casting ((<(T|undefined)[]> arr).slice(-1)[0]) to make TypeScript (2.9) do what I wanted it to do.

commented

Just wanted to bring things back to this suggestion: #13778 (comment)

This would fix the problems with indexed types that I've experienced. It'd be great if it was the default, but I understand that that would break plenty of things in the real world.

commented

To me, there's a simple solution. Enable a flag which checks for this. Then, instead of:

const array = [1, 2, 3];
for (var i =0; i< array.length; i++) {
    array[i]+=1; // array[i] is possibly undefined
}

You do:

const array = [1, 2, 3];
array.forEach((value, i) => array[i] = value + 1);

Then, when doing random index accesses, you are required to check if the result is undefined, but not while iterating an enumerated collection.

I still think this warrants having an open issue.

commented

As a programmer who is new to TypeScript, I found the situation around indexing objects in strict mode to be unintuitive. I would have expected the result of a lookup to be T | undefined, similarly to Map.get. Anyway, I just ran into this recently, and opened an issue in a library:

screepers/typed-screeps#107

I'm probably going to close it now, because it seems there's no good solution. I think I'm going to try "opting-in" to T | undefined by using a little utility function:

export function lookup<T>(map: {[index: string]: T}, index: string): T|undefined {
  return map[index];
}
commented

There were some suggestions here to explicitly set T | undefined as the object index operation return type, however that does not seem to be working:

const obj: {[key: string]: number | undefined} = {
  "a": 1,
  "b": 2,
};

const test = obj["c"]; // const test: number

This is VSCode version 1.31.1

commented

@yawaramin Make sure you have strictNullChecks enabled in your tsconfig.json (which is also enabled by the strict flag)

commented

If your use case requires arbitrary indexing on arrays of unknown length, I think it is worth adding the undefined explicitly (if only to "document" that unsafeness).

const words = ... // some string[] that could be empty
const x = words[0] as string | undefined
console.log(x.length) // TS error

Tuples work for small arrays of known lengths. Maybe we could have something like string[5] as a shorthand for [string, string, string, string, string] ?

commented

Very much in favor of this as an option. It's a noticeable hole in the type system, particularly when the strictNullChecks flag is enabled. Plain objects are used as maps all the time in JS, so TypeScript should support that use case.

commented

Ran into this with array destructing of a function parameter:

function foo([first]: string[]) { /* ... */ }

Here I'd expect a to be of type string | undefined but it's just string, unless I do

function foo([first]: (string | undefined)[]) { /* ... */ }

I don't think we have a single for loop in our code base, so I'd happily just add a flag to my tsconfig's compiler options (could be named strictIndexSignatures) to toggle this behavior for our project.

commented

This is how I've been working around this issue: https://github.com/danielnixon/total-functions/

commented

Good workaround, I really hope this gets sherlocked by the TypeScript team.

When a programmer makes assumptions in written code, and the compiler cannot deduce that it is save, this should result in a compiler error unless silenced IMHO.

This behavior would also be very helpful in combination with the new optional chaining operator.

commented

I ran into an issue with using | undefined with map today while using Object.entries().

I have an Index type that's described fairly well by {[key: string]: string[]}, with the obvious caveat that not every string key possible is represented in the Index. However, I wrote a bug that TS didn't catch when trying to consume a value looked up from the Index; didn't handle the undefined case.

So I changed it to {[key: string]: string[] | undefined} as recommended, but now this leads to issues with my use of Object.entries(). TypeScript now assumes (reasonably, based on the type specification) that the Index may have keys that have a value of undefined specified, and so assumes the result of calling Object.entries() on it may contain undefined values.

However, I know this to be impossible; the only time I should get an undefined result is looking up a key that doesn't exist, and that would not be listed when using Object.entries(). So to make TypeScript happy I either have to write code that has no real reason to exist, or override the warnings, which I'd prefer not to do.

commented

@RyanCavanaugh, I assume your original reply to this is still the TS team's current position? Both as a bump to that because nobody reads the thread and checking in case a couple more years experience with more powerful JS collection primitives, and introducing several other options to increase strictness since have changed anything.

(The examples there are still somewhat unconvincing to me, but this thread's already made all the arguments, another comment isn't going to change anything)

commented

If someone wants to modify their lib.d.ts and fix all the downstream breaks in their own code and show what the overall diff looks like to show that this has some value proposition, we're open to that data.

@RyanCavanaugh Here are some of the cases from my code base where some runtime crashes are slumbering. Note that I already had cases where I had crashes in production because of this and needed to release a hotfix.

src={savedAdsItem.advertImageList.advertImage[0].mainImageUrl || undefined}

return advert.advertImageList.advertImage.length ? advert.advertImageList.advertImage[0].mainImageUrl : ''

birthYear: profileData.birthYear !== null ? profileData.birthYear : allowedYears[0].value,

upsellingsList.upsellingProducts[0].upsellingProducts[0].selected = true

const latitude = parseFloat(coordinates.split(',')[0])

const advert = Object.values(actionToConfirm.selectedItems)[0]
await dispatch(deactivateMyAd(advert))

In this case it would be annoying as ArticleIDs extends articleNames[] includes undefined in the resulting values, while it should just allow completely defined subsets. Easily fixable by using ReadonlyArray<articleNames> instead of articleNames[].

export enum articleNames {
    WEB_AGB = 'web_agb',
    TERMS_OF_USE = 'web_terms-of-use',
}
export const getMultipleArticles = async <ArticleIDs extends articleNames[], ArticleMap = { [key in ArticleIDs[number]]: CmsArticle }>(ids: ArticleIDs): Promise<ArticleMap> => {...}

All in all I'd really like to have this extra type safety for preventing possible runtime crashes.

commented

I went into the reasoning a bit more at this comment #11238 (comment)

Think of the two types of keys in the world: Those which you know do have a corresponding property in some object (safe), those which you don't know to have a corresponding property in some object (dangerous).

You get the first kind of key, a "safe" key, by writing correct code like

for (let i = 0; i < arr.length; i++) {
  // arr[i] is T, not T | undefined

or

for (const k of Object.keys(obj)) {
  // obj[k] is T, not T | undefined

You get the second kind from key, the "dangerous" kind, from things like user inputs, or random JSON files from disk, or some list of keys which may be present but might not be.

So if you have a key of the dangerous kind and index by it, it'd be nice to have | undefined in here. But the proposal isn't "Treat dangerous keys as dangerous", it's "Treat all keys, even safe ones, as dangerous". And once you start treating safe keys as dangerous, life really sucks. You write code like

for (let i = 0; i < arr.length; i++) {
  console.log(arr[i].name);

and TypeScript is complaining at you that arr[i] might be undefined even though hey look I just @#%#ing tested for it. Now you get in the habit of writing code like this, and it feels stupid:

for (let i = 0; i < arr.length; i++) {
  // TypeScript makes me use ! with my arrays, sad.
  console.log(arr[i]!.name);

Or maybe you write code like this:

function doSomething(myObj: T, yourObj: T) {
  for (const k of Object.keys(myObj)) {
    console.log(yourObj[k].name);
  }
}

and TypeScript says "Hey, that index expression might be | undefined, so you dutifully fix it because you've seen this error 800 times already:

function doSomething(myObj: T, yourObj: T) {
  for (const k of Object.keys(myObj)) {
    console.log(yourObj[k]!.name); // Shut up TypeScript I know what I'm doing
  }
}

But you didn't fix the bug. You meant to write Object.keys(yourObj), or maybe myObj[k]. That's the worst kind of compiler error, because it's not actually helping you in any scenario - it's only applying the same ritual to every kind of expression, without regard for whether or not it was actually more dangerous than any other expression of the same form.

I think of the old "Are you sure you want to delete this file?" dialog. If that dialog appeared every time you tried to delete a file, you would very quickly learn to hit del y when you used to hit del, and your chances of not deleting something important reset to the pre-dialog baseline. If instead the dialog only appeared when you were deleting files when they weren't going to the recycling bin, now you have meaningful safety. But we have no idea (nor could we) whether your object keys are safe or not, so showing the "Are you sure you want to index that object?" dialog every time you do it isn't likely to find bugs at a better rate than not showing it all.

I do agree with the delete file dialog analogy, however I think that this analogy can also be extended to forcing user to check something that is possibly undefined or null, so this explanation don't really make sense, because if this explanation is true, the strictNullChecks option is going to induce the same behaviour, for example getting some element from the DOM using document.getElementById.

But that is not the case, a lot of TypeScript user wants the compiler to raise flag about such code so that those edge cases can be handled appropriately instead of throwing the Cannot access property X of undefined error which is very very hard to trace.

In the end, I hope these kind of features can be implemented as extra TypeScript compiler options, because that's the reason users want to use TypeScript, they want to be warned about dangerous code.

Talking about accessing array or objects wrongly is unlikely to happen, do you have any data to backup this claim? Or is it just based on arbitrary gut feeling?

commented 
for (let i = 0; i < arr.length; i++) {
  console.log(arr[i].name);

TypeScript‘s Control Flow Based Type Analysis could be improved to recognize this case to be safe and not require the !. If a human can deduce it is safe, a compiler can too.
I‘m aware that this might be non-trivial to implement in the compiler though.