In my opinion and practice, these folks aren't evaluators. They aren't evaluating their own practice, they aren't practicing evaluative thinking, and, well, I'm struggling to find the values part of our profession in their practice.

But what do I know? They have full time jobs. I barely have any work. The all-knowing market has spoken!

#RealEvalTalk #LLMs #evaluation

Sadly, there's going to be a lot of loud, enthusiastic, seemingly uncritical noises about #LLMs from evaluators at #AEA23 (American Evaluation Association). There at least 5. Only one session has a person I know will offer any concerns.

This is so frustrating and disappointing. By all means, tell on yourselves.

#RealEvalTalk

The fireworks last night (until midnight!!) did not help with the overall ambiance of my kitchen drain clearing journey, though they did cover up my swearing. Mostly.

Nothing makes me long for a complex database more than snaking a drain, although the similarities are numerous and deep. Overall though, both leave me in a cramped space, tracing connections, trying to figure out what others did and what it means - all with my ass hanging out.

#RealEvalTalk #evaluation #plumbing

Visitor Studies Association conference site: https://visitorstudies.org/2023-annual-conference

Session info (July 12 2:15-3:15pm EST) is in the image with alt text, but I'm happy to give more info if folks would like!

In sharing my part, I hope that we can discern something closer to the truth(s) we're experiencing and find some ways forward. Lord knows that isn't happening with folks silenced or staying quiet...or polite.

#evaluation #accountability #EquitableEvaluation #RealEvalTalk #spicy

@dsalo I agree! It totally does. As it does for other GLAM orgs. But wow, I've yet to see but a few really respond thoughtfully and in a meaningful way to *any* of the opportunities over the last 5 years.

Change is threatening, and interia is a hell of a drug, eh? And no one wants to pay for data projects that go beyond the ultra superficial. So, the hand wringing and flailing continues.

#RealEvalTalk #evaluation #OrganizationalCulture #OrganizationalChange

"I'm not here to bless this bullshit"

AMEN.

#RealEvalTalk

Dr. Bowman saying "they will come for you and some people won't want to work with you" because of your work against White supremacy culture in evaluation. Be prepared and find ways to nurture yourself. "Get out there and get dirty with the data and work"

I've experienced this (am experiencing this), even in my limited practice, so I can't imagine what BIPoC people experience daily.

#EViE2023 #Evaluation #CulturallyResponsiveEvaluation #RealEvalTalk

Info about Dr. Nicky Bowman:

https://www.bpcwi.com/

She brings the Real Eval Talk, too.

#EViE2023 #Conference #RealEvalTalk

Just a general note:

If you appreciate and are using the labor and IP of a consultant who was central to the work you're being interviewed about - maybe cite them? Say their name(s).

Especially if that consultant is looking for work. After there was suddenly no budget room.

Like me. The evaluator that did the evaluation and a ton of the organizational equity work their praising is me.

https://www.youtube.com/watch?v=qsPXs7xEzUQ&t=35s

#evaluation #EquitableEvaluation #RealEvalTalk

David Fetterman - who writes about empowerment evaluation no less - is one of the folks pushing "AI" and is completely clueless. Rest assured, he has a session proposal in about it. :yell: :yikes:

#RealEvalTalk #evaluation

@emilymbender @alex

More evaluators should be in this struggle against "mathy maths," SALAMIs, and "AIs." I guess that only matters if they are truly interested in practicing equity and promoting restorative justice.

I'm just tired of researching and hearing those phrases as performance.

#EquitableEvaluation #RealEvalTalk #evaluation #LLM

LB: Listened again after several frustrating conversations about LLMs in evaluation. I don't know when we'll reach critical mass in the number of folks calling bullshit on the hype and misanthropic interactions these folks are pushing.

I can only hope we sense that progress soon.

#evaluation #EquitableEvaluation #LLM #FreshHell #RealEvalTalk

H/t @emilymbender @alex

I'm not the first to say this kind of thing, but I might have been the first to say it at this organization where I had been invited as a discussant.

I don't think folks were ready for that type of introspection (maybe I'm wrong).

So my question to y'all is, how can we encourage people along those lines, towards organizational change at this foundational level?

Because of the continuing harm, it's time for action.

#EquitableEvaluation #evaluation #RealEvalTalk #OrganizationalChange

Because I am seriously done with half measures (theoretically, evaluatively, and practically), I brought up in a meeting the idea that being an ally isn't an endpoint. If we're serious about anti-racism, equitable evaluation, and liberation, we should think of ourselves as advocates, collaborators, and agitators. Allies, after all, can be seen as a political role that comes and goes. If I'm a collaborator, I'm committed to take the risk with folks.

#EquitableEvaluation #evaluation #RealEvalTalk

I don't get it.

So many decisions are already made without research/data (even broadly defined), so why bother with this fuckery?

It does give a slight gloss of...something...but even that would disappear if examined for a heartbeat.

Just pay for research and evaluation. Stop playing this game.

#RealEvalTalk

https://mastodon.lol/@schock/110145909145165332

Curious that, after eliminating evaluator positions, museums have all these evaluation internship "opportunities." Who exactly is providing mentoring?

It's extraction, plain and simple. Well, and stupidity, desperation, and willful ignorance...among other things. I feel bad for any interns that get those positions, and angry that some executives/boards chose and continue to choose this path.

#RealEvalTalk #evaluation

It really is stunning how some evaluators are sprinting to use and promote salamis "AI." Then again, when I reflect on the field and the people involved, I guess I shouldn't be surprised.

When I called some of his specific claims into question as well as foundational issues, the author of a promo piece just suggested another.

Folks in evaluation are wasting a tremendous amount of time and effort on something that introduces more distance and confusion.

#evaluation #RealEvalTalk #FigureItOut

This stuff has to stop, and unless people with the privilege to say something do so, it won't. Too many other aspects of White supremacy culture and predatory capitalism support its continuation.

It's their loss.

#RealEvalTalk #Academia #nonprofits

@curatedjenny
The study gives every indication that it wasn't participatory or equitable in any perceivable way, so it's centered even more on the organizations and staff involved.

It's reductionist, at best. Certainly exclusionary.

The graphics are nice?

This kind of thing is deeply frustrating to many evaluators that give a damn about people and recognize complexity. Like, I completely understand your (@curatedjenny) response!

#evaluation #RealEvalTalk #EvaluationFails

A bit sad that the local arts council doesn't seem to know how to collect data in responsible, non-extractive ways. I've tried to address it a few times, but eh, they don't seem that interested.

There's no relationship. They just want to show up at the drop of a hat, use a museum, and visitors. No discussion, no warning, no direction.

Hard no. And we *can* say no to funding agencies, y'all. We don't have to put up with this. If not for ourselves, stand up for the communities!

#RealEvalTalk