[Aquila]

Starting with the results:



Scores are out of 9 possible points from 9 categories, including: Display, Battery, Build & Design, Audio, SoC/System Performance, Options, Software, Security and Camera. Each category has sub categories which add up to one possible point each.

Here are the results by category:



Sub Categories and Definitions:

Display
Accuracy - Display Color Accuracy as measured by Anandtech
Brightness - Max brightness as measured by Anandtech
Quality - Takes into account screen power consumption and viewing angles
Resolution - Max resolution of the device (most of the time measuring QHD vs FHD)

Battery
Size - Battery Size
Life - Battery Life
Charging Speed - Self Explanatory
Charging Tech - Highest marks for properly implemented USB C, lowest marks for broken or proprietary implementations

Build & Design
Navigation Buttons - Highest marks for onscreen buttons, lowest marks for off-screen physical buttons (as opposed to capacitive or hybrid)
System Buttons - Accounts for location and quality of volume up/down and power buttons
Build Quality - Fit and finish – how well the device is put together, including durability considerations
Build Materials- In this ranking Metal > Glass > Plastic
Layout - Device layout, includin reachablility of components and the phone face design
Screen Size - Measures distance from 5.5” – considered by a quick survey to be the best phone size

Audio
Speaker Location - Highest marks for two stereo speakers on the front of the device, lowest marks for a speaker on the back of the device
Speaker Quality - How good the speakers on the phone sound
DAC & Amp - Added technology to make headphone listening experience better

Performance
GPU - Quality of GPU performance on SoC
CPU - Quality of CPU performance on SoC
Base Storage - Lowest Storage Option Available
Max Storage - Highest Storage Option Available
RAM - Base RAM quantity – if more than one version of phone, lowest is marked here (for example, a phone where the 64GB version comes with an extra GB of RAM).

Options
Colors - Volume of color options, highest marks for 3 or more color options
Storage Size - Highest marks for more storage options, lowest for only having one version available – No points are added for having removable storage unless Google’s Adoptable Storage is implemented
Carrier Choice - Availability on all major networks in the US – preference given to one device SKU that covers all four

Software
Resist Obsolecence - Highest software version as of today compared to highest software version predicted for EOL
Timely Updates - OEM track record for speed of firmware updates and security updates
Design - UI and OEM added app designs
Lack of Bloatware - Self Explanitory

Security
FPS - Quality of Fingerprint Scanner
Software - Security Software (level of encryption, highest security update)

Camera
DXOMark - Score on DXOMark
Options - Availability of sought-after features, such as manual controls, etc.
Consistency - Reliability of camera to get a decent or good shot every time, rather than just being capable of getting a good shot in certain conditions
Speed - Speed of camera, shutter and cycle.

[Aquila]

Many of the scores are objective and those that aren't are typically review based, however some of the things on here express what I want in a phone - such as on screen buttons being given more points than off screen. Because I weighted everything evenly as one point per category, a lot of those things wash out and that explains why the Galaxy S7 (objectively one of the best phones anywhere) is up in 2nd and 3rd place - despite the fact I'd never buy it because it fails in so many of the design preference categories I have. Hopefully decisions like that cancel out most of my bias, but here's my comparison list of the order in which I'd buy phones based solely on my preferences:

1. Nexus 6P
2. Nexus 5X (Taken out of scores because it screws things up by being too awesome in a weird package)
3. Nexus 6
4. Moto X Style
5. Moto X 2014
6. HTC 10
7. LG G4
8. LG G5
9. iPhone 6S Plus
10. Galaxy S7 Edge
11. HTC M9
12. Galaxy Note 5
13. Galaxy Note 4
14. Galaxy S6


Device ranks in scores:

1. Nexus 6P
2. Galaxy S7 Edge
3. Galaxy S7
4. HTC 10
5. iPhone 6S Plus
6. Galaxy Note 5
7. LG G5
8. Galaxy S6
9. Nexus 6
10. Galaxy Note 4
11. Moto X Style
12. LG G4
13. HTC M9

[Aquila]

While awaiting further details from Anandtech, I've got preliminary results that include the Moto Z and the OnePlus 3 and the camera score on the G5 went up due to their rating on DXOMark today.

This created several rank changes, including the Note 4 pushing past the Nexus 6 (and pushing the Nexus 6 out of the top 10) and the S6 pushing above the LG G5.




I anticipate the Moto Z going up somewhat when it's reviews are complete because it has a lot of assumed values that match the Moto X Style and in those areas it is very likely to be better.

[Aquila]

Anandtech's OnePlus 3 review came out today - it was very bad news for the OP3, which due to a worse than expected performance on memory management, very bad display, battery and charging tech caused the OP3 to move from 9th place to 13th. The ultrabad display ended up pushing the Nexus 6 from 2014 back up a few points, raising its score from 6.89 to 6.97 - which was enough to bounce all the way back up to that 9th place position.

[Aquila]

Full scores

[Aquila]

Alright, major update to the methodology today. Four sub categories have been removed because they are too subjective. Those were layout, system buttons, navigation buttons and software design.

Included a bonus point (which amounts to a .04 score increase) for having a micro SD card. Right now that's the really real thing separating the rumored Marlin and Note 7.

Added four new devices today and did some major upgrading to the OP3 based on it's new data from Anand. Also gave Samsung a bump on software updates because of their recent performance on the S7 series. Unfortunately, the 4 new devices and the Moto Z are not actually out yet - so they're definitely still in flux. This meant adding another column to the rankings - Real Phone Rank - which ranks them in order of devices that actually exist and can be reviewed.

[Aquila]

Removed "color choices" and "speaker location" as categories, added some known facts to replace assumptions and normalized camera, variance in size, storage size choices, carrier choices, and update assumptions.



So finding out the Note 7 is probably 6GB RAM, etc. pushed it down from the king of the rumored phones to 2nd place behind the Marlin. S7 Edge retained it's best real phone ranking and the 6P, my personal favorite, dropped from 2nd to 3rd real phone, being passed by the regular S7.

Axon 7 will likely move up further once people have it and we get some solid reviews.

OnePlus 3 moved ahead of the Nexus 5X on real phones but stayed at the same rank because the Moto Z Force moved up based on some updated data.

Two weeks until we lock in most of the assumptions about the Note 7.

[Aquila]

Full data including rumoreds

[yeahizme]

No love for the V10???

[Aquila]

No love for the V10???

Good point; I don't know enough about it but I can research and get it added in there if there is enough data.

[yeahizme]

Ok cool :-)

[Aquila]

The Note 7 is real now, so we get an update:

[anon(9072051)]

So the Nexus 6 gets a 50% security score even though it shipped encrypted by default?

[Aquila]

So the Nexus 6 gets a 50% security score even though it shipped encrypted by default?

I believe it got 100% for software and 0% for extra hardware features for an average of 50% but I can double check that.

[anon(9072051)]

I believe it got 100% for software and 0% for extra hardware features for an average of 50% but I can double check that.

Nevermind. I hadn't read the start of post #2 before.

[Aquila]

Nevermind. I hadn't read the start of post #2 before.

Since that time everything subjective has been removed.