Canvas is down as ShinyHunters threatens to leak schools’ data

Posted by stefanpie 13 hours ago

688 points | 418 commentspage 2

matthewfcarlson 12 hours ago|

I remember circa 2010 a friend of mine at college was like “blackboard sucks, let’s build something new”. At the time I poo pood the idea and lo and behold canvas came out a year later. Outside looking in, they been crushing it.

HPMOR 11 hours ago||

One of my mentors created Blackboard. It used to be very very good, but he sold it to private equity, and they immediately fired all of the customer support and developers, 3xd prices overnight leading to the 'blackboard sucks' problem. This gave the opening for Canvas to eventually come on to the scene and dominate.

corvad 10 hours ago|||

I believe Canvas was also sold to private equity pretty recently too. https://www.instructure.com/press-release/instructure-to-be-...

whoahwio 9 hours ago||

canvas was bought by PE for the first time in 2020 https://www.thomabravo.com/portfolio/instructure

rolandog 11 hours ago||||

My wife and I each have to use it as we're both following an online master's at the same university... it's definitely gone downhill (compared to the days where I originally used it ~20 yrs ago in college; tracker-riddled, slow); surprisingly, a recent change made it so that you can only attend online lessons in Chrome (haven't had time to see if this is just a user-agent thing).

redwood 10 hours ago|||

..and be acquired by PE so the cycle can continue.. https://www.instructure.com/press-release/instructure-to-be-... sigh. Barbarians at the gate probably didn't double down on security

moduspol 11 hours ago|||

I worked in a college IT department around that time and the common belief was that all LMSes suck. There are just too many different ways that too many different people want to do things that it's just bound to be hated. Kind of like Jira / Asana for software dev project management.

SamuelAdams 10 hours ago||

LMS’s are a lot like programming languages. There’s the ones people complain about and the ones no one uses.

asdff 11 hours ago|||

I used both and could not tell you the major differences. I feel like they are equivalent in the bread and butter features. Most people don't use 99% of the functions they bake into these. Just use it to hold the syllabus, maybe hold the slides, submit assignments, and spreadsheet for grades. All stuff you can do with email + spreadsheet already. Maybe throw in a shared drive for larger files, which every university in the country already pays for.

quadrature 11 hours ago|||

"Equivocal describes something ambiguous, uncertain, or open to multiple interpretations, often used to intentionally mislead or evade."

do you mean equivalent ?.

asdff 11 hours ago||

yes

vlunkr 10 hours ago|||

Blackboard got a lot better in response to the flood of customers heading to canvas.

kayyyy 11 hours ago|||

As someone who has used both as a student and a TA I find blackboard miles better, much easier to find what i'm looking for and my professors seem to have better luck laying out their course on blackboard than canvas.

breakingstuff 10 hours ago||

I actually disagree, based on my time using Blackboard as an admin, student, and teacher. Although my experience is a few years out of date, I found the interface cumbersome and the performance slow.

russfink 6 hours ago||

It depends on what vintage of Blackboard your IT team has installed. We moved from a circa 2011 BB instance to Canvas in 2022, and it was hands down superior. A different university is running the most recent BB and it’s similar to Canvas.

JumpCrisscross 8 hours ago|||

> circa 2010

Instructure, "the developer and publisher of Canvas," was founded in 2008 [1].

[1] https://en.wikipedia.org/wiki/Instructure

jer0me 6 hours ago||

That sounds like “circa 2010” to me. And Canvas was launched in 2011, according to the article you linked.

smurda 10 hours ago|||

Blackboard, the Canvas predecessor, was so unstable that we called it BlackOutBoard

ramon156 4 hours ago|||

How does canvas compare to Brightspace?

brandonmenc 8 hours ago|||

Maybe schools should be self-hosting something like Sakai instead.

forgetfreeman 10 hours ago||

They are definitely crushing it on sales. The actual product is a radioactive dumpster fire that is simultaneously hostile to students, teachers, and parents.

dghlsakjg 10 hours ago||

Yeah but the customer is the administrators who never have to make contact with the real world

exprez135 15 hours ago||

The Canvas instance at the nearby university is now down (May 7, 4 PM Eastern), but was briefly displaying the message in this screenshot (1). The ransom message implies that today's problem is the second wave in an attack on Instructure after ignoring their first breach in recent days.

1: https://ibb.co/r29RjdnH

HDBaseT 13 hours ago|

Yeah, this is ongoing.

We received communication that Canvas is down for "Under Maintenance" although it seems ShineyHunters have compromised Canvas again with that message you posted.

We do not see that message anymore, although all instrucuture.com URLs are down. The list of schools in the ShinyHunters publication can be found here: https://web.archive.org/web/20260507042014/http://91.215.85....

GaryBluto 7 hours ago|||

https://web.archive.org/web/20260507042014fw_/http://91.215....

Original now shows 404.

nebula8804 12 hours ago|||

Seems like Canvas instances of schools not listed are also down (at least my alma mater is)

goldenskye 12 hours ago|||

Yes, I work for an Australian online school. We’re down “for scheduled maintenance” (I question how “scheduled” it was given this is within school hours on a school day), but we’re not on the list published by ShinyHunters.

avs733 11 hours ago|||

our instance went from [insert hacker leet text] to "down for scheduled maintenance" and myself and other faculty are just having the darkest humor about this.

HDBaseT 12 hours ago|||

[dead]

HDBaseT 12 hours ago|||

[dead]

sharkweek 12 hours ago||

My wife is in grad school at a major university and is dealing with this right now the week of midterms for spring quarter.

I totally understand why a university wouldn’t want to bake their own learning portals but just feels like such a single point of risk to use third party solutions for something like this.

Back in my day… all we had was a school email via on-premise services. I guess we registered for classes in a web portal but that’s about it. The idea of online class was entirely foreign at the time. Ain’t nobody hacking a blue book.

gdhkgdhkvff 10 hours ago||

It’s wild to me that people in this comment section are suggesting that schools should improve their security by rolling their own platform, which is bound to be filled with security holes, instead of using a popular, maintained, open source option.

nazgul17 10 hours ago|||

To be fair to the idea, though, while this would make individual instances less secure, it would drastically decrease the leverage for the work bad actors put in.

There is a saying in the software security industry that (I'm paraphrasing from rusty memories) a system is secure if the cost of hacking it is higher than the value it protects.

Each system being completely distinct from another means that the cost of hacking the average student goes up by 9000 (from the article, Canvas is used by 9000 schools).

Still not saying that rolling out your own is the preferred solution, but the idea is not as ludicrous as it would seem, and should definitely be entertained and discussed, at least.

forgetfreeman 10 hours ago|||

Maybe. I still remember the Drupal community sneering at the New York Times when they unveiled their homegrown online news platform bitd. After 15 years of recursively scraping ad-hoc porn sites off of server hard drives when clients dragged their feet on migrating to latest versions I 'm less certain the assumption that homegrown == less secure is as valid as it sounds.

shnock 4 hours ago||

Could you explain the last sentence a bit more? I don’t follow

asdff 11 hours ago|||

Universities used to do this sort of stuff themselves. Then it became a business handled by purchasing rather than needs met by the department themselves.

afavour 11 hours ago|||

In fairness in the era where universities did it themselves the tech requirements and expectations were dramatically lower.

asdff 11 hours ago|||

Tech requirements are the same as they always were. One needs to ask whether they need so many frameworks to host some files on the internet and submit some files and perform spreadsheet calculations. We still used one of those First Age 1990s websites for sort of pre lab quizzes this one class when I was going through it, and it might have looked a little "old" but I mean it did the thing and worked for years and will continue to do the thing and work for years.

internetter 10 hours ago||

You're being deliberately obtuse. Canvas has many many features. Wikis and discussion boards and quizzes (with some anticheat) and groups and the list goes on and on. Furthermore, while it was never the flashiest thing, it did it better than many of its predecessors. Yes, an individual class may not use all of these features, and yes canvas has suffered feature creep even over my time as a student and yes canvas is not doing anything technically challenging, but there is enough of it that each school rolling their own everything would be a drastic waste of everybody's time and money.

clipsy 11 hours ago|||

Have these dramatically higher tech requirements and expectations improved the quality of education whatsoever?

avs733 11 hours ago|||

Because faculty didn’t want to do it anymore. They want it handled by others but also they want oversight and veto power but also they don’t want to be bothered. But it better always work, and if they make a mistake the software is broken because don’t tell them it’s a user error they used to write Fortran.

As a faculty member at a large university…I have a deep respect for the impossible job of university IT departments.

We originally rolled our on LMS decades ago. When we switched to canvas we kept the home brew running for five years past its expiration date because faculty refused to remove their files. Finally each one was manually moved by IT for the recalcitrant old faculty.

asdff 11 hours ago||

It is kind of funny when these LMS tools with 100+ functions are being used for little more than what email, a grades spreadsheet, and maybe a shared drive would do. University might even ask for the final grades in spreadsheet format by the end of the term anyhow, so data goes into the LMS just to come back out again.

avs733 10 hours ago||

In a sense you aren’t wrong but those analogies fail at scale. It’s like saying you could replace all hr functions with a spreadsheet.

They are large databases yes but they do a lot of small and large things that that analogy glosses over

jagged-chisel 11 hours ago|||

> Ain’t nobody hacking a blue book.

Well not with that attitude

walrus01 10 hours ago|||

A university doesn't need to bake its own learning portal, Moodle exists and is used by a lot of large schools.

ibgeek 11 hours ago|||

Moodle is an open-source LMS that can be self-hosted.

https://moodle.org/

hoppyhoppy2 11 hours ago||

Another open-source LMS that can be self-hosted is... Canvas.

wmoxam 10 hours ago|||

Almost no one does

ibgeek 10 hours ago|||

Didn't realize that. Thanks for the info!

userbinator 11 hours ago||

I totally understand why a university wouldn’t want to bake their own learning portals

They used to, in the pre-cloud/SaaS era; and they were much simpler and better UX than the slop that they're renting today, because the actual users were not far from the developers.

oezi 9 hours ago||

Counterpoint: I was a PhD student in 2004 and on the universities board* which oversaw the roll-out of the campus management system. It cost > 10m EUR to implement a shitty system with the worst UX and years of stabilizing to make it somewhat work.

The amount of corner cases and performance requirements during rush times (semester start) made it really infeasible for a university to roll their own.

* German universities have this funny system where 51% of such boards are controlled by the professors and the rest is made up of other employees/staff and students. They call it academic participation.

cocoacat 13 hours ago||

Also here: https://news.ycombinator.com/item?id=48054386

somebudyelse 11 hours ago||

It looks like Instructure has been removed from the ShinyHunters website. Both the entry and the list of schools has been removed.

bombcar 11 hours ago||

Look for large BTC moves recently?

corvad 10 hours ago||

Ransom paid?

tom1337 13 hours ago||

> Canvas is currently undergoing scheduled maintenance

doesn't seem that scheduled to me

javawizard 12 hours ago||

ex-Instructure employee here (though it's been about 10 years since I worked for them).

That's just the quickest page/status update to throw up; it was a one-liner to push it live back when I was on the deploy rotation.

I'd hazard a guess they have more important things to worry about right now than exact status page messaging ;)

chrisjj 2 hours ago||

> That's just the quickest page/status update to throw up

Funny how a lie is always quicker than the truth...

podiki 8 hours ago|||

I thought the same. The "scheduled" part of the message is gone now, at least on the instance I use.

anematode 12 hours ago||

Well, scheduled by whom? :)

mystraline 12 hours ago|||

Whoever it is, is likely defended by Cloudflare. They seem to like the booters.

https://news.ycombinator.com/item?id=48025001

SeanAnderson 8 hours ago||

https://status.instructure.com/ implies Canvas became available again about thirty minutes ago from the time of this post.

Is this accurate? Or is this still an ongoing issue?

podiki 8 hours ago||

Ongoing. It is not "down" but purposefully offline for "maintenance." Main status does show the LMS (all the course stuff) down, and my instance shows "up" but that's because (I assume) you can reach it and the maintenance page. But that's not useful, if technically not "down."

SeanAnderson 8 hours ago||

Thanks

boldi 7 hours ago|||

Canvas LMS is the core service that universities rely on. I assume they're trying to develop a fix and that's why the service is labeled "Under Maintenance". I'm a Berkeley student and can confirm that our instance (bcourses.berkeley.edu) is still down.

owlboy 5 hours ago||

Federated logins appear to now be broken for the campus I’m affiliated with. So more action is needed.

incomplete 14 hours ago||

yep, i work for a major university and our canvas instance is down. this is really, really bad.

edit: here's the list of impacted universities (unsure if they all have their canvas instances offline, but i'd be surprised if not): http://91.215.85.103/pay_or_leak/instructure_affected_school...

starkrights 12 hours ago||

The source txtfile has since either been dos'd or deleted (at least it was when I tried to access)

Someone dumped the content into a google doc on reddit[1] if anyone's interested.

[1]: https://docs.google.com/document/d/1MTktVSwTUM5I_w7bKNGj94sT...

rigrassm 8 hours ago||

> The source txtfile has since either been dos'd or deleted (at least it was when I tried to access)

> Someone dumped the content into a google doc on reddit[1] if anyone's interested.

> [1]: https://docs.google.com/document/d/1MTktVSwTUM5I_w7bKNGj94sT...

Thanks for linking this. Ended up finding my kids school district on the list unfortunately.

12_throw_away 13 hours ago|||

tbh this has me wondering if canvas "instances" are actually as isolated and segregated from each other as they're supposed to be.

javawizard 11 hours ago|||

Define "as they're supposed to be".

Back when I worked for Instructure ~10 years ago, Canvas was effectively a single, giant, monolithic multitenant app with one instance backed by several thousand app servers and ~100 separate Postgres database clusters that any app server could talk to.

Schools were grouped onto pools of app severs and Postgres database clusters more or less according to locality and cluster availability. I want to say a handful of the largest schools got their own clusters, but I'm not certain, and at any rate their clusters could certainly all talk to each other.

It was actually kind of neat from a technical perspective: any Rails model across the entire Canvas world could have a "foreign key" pointing to any other Rails model anywhere else. Among other things, this allowed for users who could administer multiple Canvas organizations, even if those organizations resided on different Postgres clusters. https://github.com/instructure/switchman is their gem that made that all work. (I put "foreign key" in quotes because the whole thing was implemented in software, not with actual database FKs, for obvious reasons.)

---

Of course, the massive downside to that sort of thing is that if you manage to pop one Canvas app server, you have the keys to the kingdom. I wonder if they'll sharpen the edges between clusters in response to this...

---

(Disclaimer: I left Instructure back in 2017; much could have changed since then, and my memory could be faulty about the specifics. Caveat emptor.)

wky 13 hours ago||||

It's possible that Instructure's servers got compromised:

dig canvas.ucdavis.edu

    [...]
    
    ;; ANSWER SECTION:
    canvas.ucdavis.edu. 1974 IN CNAME ucdavis-vanity.instructure.com.
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.125
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.103
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.15
    ucdavis-vanity.instructure.com. 60 IN A 18.173.121.18

dig canvas.duke.edu

    ;; ANSWER SECTION:
    canvas.duke.edu. 300 IN CNAME duke-vanity.instructure.com.
    duke-vanity.instructure.com. 60 IN A 18.173.121.125
    duke-vanity.instructure.com. 60 IN A 18.173.121.18
    duke-vanity.instructure.com. 60 IN A 18.173.121.103
    duke-vanity.instructure.com. 60 IN A 18.173.121.15

mrsvanwinkle 13 hours ago||

that's what the screenshot says. They rooted Instructure servers.

SamuelAdams 11 hours ago|||

It depends on what you pay for. If you need FedRamp or IL4+ compliance you are likely on dedicated infrastructure. Everyone else uses multi tenancy.

GaryBluto 7 hours ago|||

https://web.archive.org/web/20260507042014fw_/http://91.215....

Cider9986 10 hours ago|||

Here's an archive https://archive.is/eB2hE

mrsvanwinkle 13 hours ago||

[dead]

tptacek 10 hours ago||

The boy is a biochem PhD student at UIUC and reports that all their finals are now cancelled. "Is this good news?" I ask. "Yes. Everything coming up Milhouse."

robertritz 10 hours ago|

I'm shocked universities don't host their own LMS? At least large universities have the IT departments to do this. They host compute clusters, so they can certainly host an LMS.

oezi 9 hours ago|

The same reason hospitals don't have their own Patient Information System but all use Epic. The amount of customization you need and continuous churn due to changing curricula and regulatory requirements makes it hard to keep up without scale.

More comments...